Tez 구성 매뉴얼(Hive는 설치되어 있다는 가정 하에). Tez 0.10.2 · Hive 3.1.3 · JDK 8 · Hadoop 3.3.3 기준(2022년 테스트 됨). 현행은 Tez 0.10.5 (2025-05-30) · Hive 4.2.1 · Java 21 이고, 빌드 절차 자체는 같다.[1]
Docker compose를 사용해 빌드 환경을 구성한다.
버전 호환성
centos 7.8
jdk 8
-maven 3.x+- 3.0.5 버전 사용시 에러로 인한 버전 변경
maven 3.3.9
protobuf 2.5
docker-compose.yml
version: '3.8'
services:
centos:
image: centos:7
container_name: tez_builder
restart: always
command: tail -F anything
volumes:
- ./source:/source
[haedong@docker ~]$ sudo docker compose up -d
[haedong@docker ~]$ sudo docker exec -it tez_builder bash
[root@1afcd511bf10 /]# cd /source
[root@1afcd511bf10 source]# yum install -y protobuf protobuf-devel bzip2 git wget java-1.8.0-openjdk-devel
[root@1afcd511bf10 source]# wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.3.9/apache-maven-3.3.9-bin.tar.gz
[root@1afcd511bf10 source]# tar xvfz apache-maven-3.3.9-bin.tar.gz
[root@1afcd511bf10 source]# vi /etc/profile
MAVEN_HOME=/source/apache-maven-3.3.9
PATH=$MAVEN_HOME/bin:$PATH
[root@1afcd511bf10 source]# source /etc/profile
[root@1afcd511bf10 source]# echo '{ "allow_root": true }' > ~/.bowerrc
https://github.com/Teleborder 해당 깃 레포 폐쇄로 라이브러리 설치 불가 (에러 내용 아래와 같음)[ERROR] bower file-saver.js#1.20150507.2 retry Download of https://github.com/Teleborder/FileSaver.js/archive/b7cf622909258086bc63ad764d08fcaed780ab42.tar.gz failed with EHTTP, trying with git..
[INFO] bower file-saver.js#1.20150507.2 checkout b7cf622909258086bc63ad764d08fcaed780ab42
[ERROR] bower file-saver.js#1.20150507.2 ECMDERR Failed to execute "git clone https://github.com/Teleborder/FileSaver.js.git /tmp/root/bower/e75d8091393c079c0030df2840479819-4641-0sZ2cE --progress", exit code of #128 remote: Support for password authentication was removed on August 13, 2021. remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication. fatal: Authentication failed for 'https://github.com/Teleborder/FileSaver.js.git/'
[ERROR]
[ERROR] Additional error details:
[ERROR] remote: Support for password authentication was removed on August 13, 2021.
[ERROR] remote: Please see https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
[ERROR] fatal: Authentication failed for 'https://github.com/Teleborder/FileSaver.js.git/'
[root@1afcd511bf10 source]# wget https://github.com/apache/tez/archive/refs/tags/rel/release-0.10.2.tar.gz
[root@1afcd511bf10 source]# tar xvfz release-0.10.2.tar.gz
[root@1afcd511bf10 source]# cd tez-rel-release-0.10.2
[root@1afcd511bf10 tez-rel-release-0.10.2]# mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true
tez-0.10.2.tar.gztez-0.10.2-minimal.tar.gz[root@1afcd511bf10 tez-rel-release-0.10.2]# ls -al tez-dist/target/
drwxr-xr-x. 2 root root 6 Aug 30 08:42 archive-tmp
drwxr-xr-x. 2 root root 28 Aug 30 08:43 maven-archiver
drwxr-xr-x. 3 root root 22 Aug 30 08:42 maven-shared-archive-resources
drwxr-xr-x. 3 root root 4096 Jan 22 2020 tez-0.10.2
drwxr-xr-x. 3 root root 4096 Jan 22 2020 tez-0.10.2-minimal
-rw-r--r--. 1 root root 17565475 Aug 30 08:43 tez-0.10.2-minimal.tar.gz
-rw-r--r--. 1 root root 67685528 Aug 30 08:43 tez-0.10.2.tar.gz
-rw-r--r--. 1 root root 2794 Aug 30 08:43 tez-dist-0.10.2-tests.jar
tez-0.10.2-minimal.tar.gz 파일을 Hive server로 전송한다.[root@1afcd511bf10 tez-rel-release-0.10.2]# scp tez-dist/target/tez*tar.gz hive@{Hive server}:/home/hdfs/tez/
[hdfs@hive-server igkim]$ tar xvfz tez-0.10.2-minimal.tar.gz
[hdfs@hive-server ]$ mkdir tez-0.10.2-minimal/conf
[hdfs@hive-server ]$ vim tez-0.10.2-minimal/conf/tez-site.xml
<configuration>
<property>
<name>tez.lib.uris</name>
<value>${fs.defaultFS}/apps/tez-0.10.2/tez-0.10.2.tar.gz</value>
</property>
</configuration>
[hdfs@hive-server ]$ vim ${HIVE_HOME}/conf/hive-env.sh
TEZ_HOME=/home/hdfs/tez-0.10.2-minimal
export HADOOP_CLASSPATH=${TEZ_HOME}/conf:${TEZ_HOME}/*:${TEZ_HOME}/lib/*
tez-0.10.2.tar.gz 파일 HDFS에 업로드[hdfs@hive-server ]$ hdfs dfs -mkdir -p /apps/tez-0.10.2
[hdfs@hive-server ]$ hdfs dfs -put tez-0.10.2.tar.gz /apps/tez-0.10.2/
[hdfs@hive-server ]$ hdfs dfs -put tez-0.10.2-minimal.tar.gz /apps/tez-0.10.2/
[hdfs@hive-server ]$ hdfs dfs -put tez-0.10.2-minimal/conf/tez-site.xml /apps/tez-0.10.2/
[hdfs@datanode-01 ]$ hdfs dfs -get /apps/tez-0.10.2/tez-0.10.2-minimal.tar.gz ./
[hdfs@datanode-01 ]$ hdfs dfs -get /apps/tez-0.10.2/tez-site.xml ./
[hdfs@datanode-01 ]$ tar xvfz tez-0.10.2-minimal.tar.gz
[hdfs@datanode-01 ]$ mkdir -p tez-0.10.2-minimal/conf
[hdfs@datanode-01 ]$ mv tez-site.xml tez-0.10.2-minimal/conf/
[hdfs@hive-server igkim]$ hive
Hive Session ID = 35e3a597-0331-4960-a6bb-a3c8b3ff976d
hive> SET hive.execution.engine=tez;
hive> SET hive.tez.container.size=4096;
hive> use igkim;
OK
Time taken: 0.99 seconds
hive> SELECT a.id, a.name, b.age FROM user_name a left join user_age b on a.id=b.id;
Query ID = hdfs_20220902112047_7067cc14-4710-45e6-87df-d5013ec95c57
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1661906760314_0030)
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
----------------------------------------------------------------------------------------------
Map 2 .......... container SUCCEEDED 1 1 0 0 0 0
Map 1 .......... container SUCCEEDED 1 1 0 0 0 0
----------------------------------------------------------------------------------------------
VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 5.33 s
----------------------------------------------------------------------------------------------
OK
2 wsjeong 31
1 igkim 34
Time taken: 15.174 seconds, Fetched: 2 row(s)
tez-0.10.2-minimal/lib 경로에 해당 라이브러리를 넣어준다.
hadoop-common-3.3.1.jarhtrace-core4-4.1.0-incubating.jar[hdfs@hadoop02 hive]$ hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/igkim/tez-0.10.2-minimal/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/hadoop-3.3.3/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
which: no hbase in (/home/hdfs/jdk/bin:/home/hdfs/hadoop/bin:/home/hdfs/hive/bin:/home/hdfs/nifi/bin:/home/hdfs/zookeeper/bin:/usr/local/bin:/bin:/usr/bin:....
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/igkim/tez-0.10.2-minimal/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/hadoop-3.3.3/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Hive Session ID = 372054f7-b336-475c-a7b7-b6f82fef4e4a
Logging initialized using configuration in jar:file:/home/hdfs/apps/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.fs.FsTracer.get(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/Tracer;
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:323)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:308)
at org.apache.hadoop.hdfs.DistributedFileSystem.initDFSClient(DistributedFileSystem.java:201)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:186)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3469)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:288)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:624)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:591)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:747)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/igkim/tez-0.10.2-minimal/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/hadoop-3.3.3/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
which: no hbase in (/home/hdfs/jdk/bin:/home/hdfs/hadoop/bin:/home/hdfs/hive/bin:/home/hdfs/nifi/bin:/home/hdfs/zookeeper/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/hom
e/hdfs/.local/bin:/home/hdfs/bin)
2022-08-31 14:27:24: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hdfs/igkim/tez-0.10.2-minimal/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hdfs/apps/hadoop-3.3.3/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/htrace/core/Tracer$Builder
at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3460)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:174)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3574)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3521)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:540)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at org.apache.hadoop.hive.common.FileUtils.getJarFilesByPath(FileUtils.java:1006)
at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:5201)
at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:5102)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:97)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:81)
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1143)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 19 more
bower alasql#^0.4.0 CERT_HAS_EXPIRED Request to https://registry.bower.io/packages/alasql failed: certificate has expired
라는 오류가 빌드 중 발생할 시
vi tez-ui/pom.xml
이 부분을
<allow-root-build>--allow-root=false</allow-root-build>
이렇게 수정해주세요
<allow-root-build>--config.strict-ssl=false --allow-root=false</allow-root-build>
Apache Tez Releases · Tez Install. 2026-09-17 확인. https://tez.apache.org/releases/index.html · https://tez.apache.org/install.html ↩︎