아래에서 JDK, SPARK 버전 호환 확인할 수 있습니다.
JDK Compatibility
Info: JavaScript is currently disabled, code tabs will still work, but preferences will not be remembered. Scala’s primary platform is the Java Virtual Machine (JVM). (Other supported platforms: Scala.js, Scala Native.) Sometimes new JVM and JDK (Java De
docs.scala-lang.org
1. Java 1.8.0 설치
GitHub - ojdkbuild/ojdkbuild: Community builds using source code from OpenJDK project
Community builds using source code from OpenJDK project - GitHub - ojdkbuild/ojdkbuild: Community builds using source code from OpenJDK project
github.com
2. Python 3.11.2 설치
Download Python
The official home of the Python Programming Language
www.python.org
- 설치시에 환경변수 추가 옵션 체크
3. Scala 3.2.3 설치
Install
To install Scala, it is recommended to use cs setup, the Scala installer powered by Coursier. It installs everything necessary to use the latest Scala release from a command line: If you are just beginning your journey with Scala, we recommend that you rea
www.scala-lang.org
- cs setup 설치
- 설치 후 껐다 키면 scala -version 시 확인 가능
4. Spark 설치
Downloads | Apache Spark
Download Apache Spark™ Choose a Spark release: Choose a package type: Download Spark: Verify this release using the and project release KEYS by following these procedures. Note that Spark 3 is pre-built with Scala 2.12 in general and Spark 3.2+ provides
spark.apache.org
5. Winutils 설치
- Spark 다운로드할 때 선택한 하둡 버전에 맞춰 winutils.exe 파일만 다운로드
GitHub - cdarlint/winutils: winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows
winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows - GitHub - cdarlint/winutils: winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows
github.com
- Hadoop\bin 폴더를 생성하여 해당 파일 저장
환경변수 설정
- bin 폴더를 가지고 있는 것들을 HOME으로 저장
- JAVA_HOME
- SPARK_HOME
- HADOOP_HOME
- Path에 추가
- %JAVA_HOME%\bin
- %SPARK_HOME%\bin
- %HADOOP_HOME%\bin
'개발' 카테고리의 다른 글
| [ASP.NET] Visual Studio 템플릿 만들기 (0) | 2022.04.05 |
|---|---|
| Visual Studio 솔루션 이름 변경하기 (0) | 2022.03.04 |
| 네이버 부스트캠프 웹모바일 6기 합격 후기 (전공자) (0) | 2021.07.13 |
| SSAFY 6기 합격후기 (전공자) (0) | 2021.06.30 |
| [DB] SQL 문법 (0) | 2021.04.12 |
최근댓글