Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
__pycache__		__pycache__
conf		conf
lib		lib
test_data		test_data
Jenkinsfile		Jenkinsfile
Pipfile		Pipfile
Pipfile.lock		Pipfile.lock
README.md		README.md
log4j.properties		log4j.properties
main.py		main.py
sbdl_main.py		sbdl_main.py
sbdl_submit.sh		sbdl_submit.sh
test_nutter_sbdl.py		test_nutter_sbdl.py
test_pytest_sbdl.py		test_pytest_sbdl.py

Repository files navigation

SBDL

About

Spark Bulk Data Load. This is a data engineering project. Ingest the data from Hive tables present in the Master Data Management Platform and do the all processing in spark.

jenkins spark data-engineering data-processing hive-table

Readme