[Recent Interests] Data Engineering / MLOps
[hadoop] yarn demon ์‹คํ–‰์‹œ java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/s
2024-02-12 01:56:42,265 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollectorManager
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$100(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethods(Unknown Source)
        at com.google.inject.spi.InjectionPoint.getInjectionPoints(InjectionPoint.java:688)
        at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:380)
        at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java:399)
        at com.google.inject.internal.BindingBuilder.toInstance(BindingBuilder.java:84)
        at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebApp.setup(RMWebApp.java:56)
        at org.apache.hadoop.yarn.webapp.WebApp.configureServlets(WebApp.java:160)
        at com.google.inject.servlet.ServletModule.configure(ServletModule.java:55)
        at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
        at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
        at com.google.inject.spi.Elements.getElements(Elements.java:110)
        at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
        at com.google.inject.Guice.createInjector(Guice.java:96)
        at com.google.inject.Guice.createInjector(Guice.java:73)
        at com.google.inject.Guice.createInjector(Guice.java:62)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:356)
        at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:401)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1137)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1245)
        at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1446)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollectorManager
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        ... 36 more
2024-02-12 01:56:42,270 INFO resourcemanager.ResourceManager: Transitioning to standby state
2024-02-12 01:56:42,270 INFO resourcemanager.ResourceManager: Transitioned to standby state
2024-02-12 01:56:42,271 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:

 
์œˆ๋„์šฐ ํ™˜๊ฒฝ์—์„œ ํ•˜๋‘ก ์„ค์น˜ ํ›„ yarn demon์„ ์‹คํ–‰ํ–ˆ์„๋•Œ ์œ„์™€ ๊ฐ™์€ ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•˜์˜€์Šต๋‹ˆ๋‹ค.
 
์œˆ๋„์šฐ cmd ์ฐฝ์—์„œ java -version ์œผ๋กœ ์ •์ƒ์ ์œผ๋กœ ์„ค์น˜๋˜์–ด์žˆ๋Š”์ง€ ํ™•์ธ. (1.8.x_xxx)
 
[ํ•˜๋‘ก๊ฒฝ๋กœ]/share/hadoop/yarn/timelineservice์˜
hadoop-yarn-server-timelineservice-3.0.3.jar ํŒŒ์ผ์„
 
[ํ•˜๋‘ก๊ฒฝ๋กœ]/share/hadoop/yarn์œผ๋กœ ๋ณต์‚ฌํ•œ๋‹ค.

(ex) C:/hadoop-3.0.3/share/hadoop/yarn/timelineservice/hadoop-yarn-server-timelineservice-3.0.3.jar => copy to C:/hadoop-3.0.3/share/hadoop/yarn
 
๋ณต์‚ฌ ํ›„ ๋‹ค์‹œ start-yarn.cmd๋ฅผ ์‹คํ–‰ํ•ด๋ณด๋ฉด ์ •์ƒ์ ์œผ๋กœ ๊ฒฝ๋กœ๋ฅผ ์ธ์‹ํ•œ๋‹ค.

 
 
reference
NoClassDefFoundError org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollectorManager