Skip to content

Commit

Permalink
[Oracle] Add output messages for Oracle during extraction (#27)
Browse files Browse the repository at this point in the history
* Flag version added

* Oracle paths fixed

* "Mobilize" removed
  • Loading branch information
sfc-gh-ibarquerogarcia authored Feb 28, 2024
1 parent e5ebc4a commit b14b74c
Show file tree
Hide file tree
Showing 12 changed files with 179 additions and 11 deletions.
4 changes: 3 additions & 1 deletion BigQuery/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This repository offers a collection of straightforward scripts designed to facil

## Version

Version 1.0 Release 2021-12-02
Release 2023-02-28

## Usage

Expand Down Expand Up @@ -35,6 +35,8 @@ The following are the steps to execute the DDL Code Generation. They can be exec
- Finally, run `create_ddls.sh` to extract the DDLs from BigQuery
- After a successful run, remove region information from the top line of `create_ddls.sh`.

3. Run `create_ddls.sh --version` to check the current version of the extraction scripts.

### DDL Files
These files will contain the definitions of the objects specified by the file name.

Expand Down
10 changes: 10 additions & 0 deletions BigQuery/bin/create_ddls.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
#!/bin/bash

#This version should match the README.md version. Please update this version on every change request.
VERSION="Release 2024-02-28"

export versionParam=$1

if [ "$versionParam" = "--version" ]; then
echo "You are using the $VERSION of the extraction scripts"
exit 1
fi

REGION='us'

echo " "
Expand Down
5 changes: 3 additions & 2 deletions DB2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@ This repository provides some simple scripts to help exporting your DB2 code so

## Version

Version 1.2
Release 2022-05-27
Release 2023-02-28

## Usage

Expand All @@ -25,6 +24,8 @@ That variable will determine if there are any database that you want to exclude

2 - After modifying, the `create_ddls.sh` file can be run from the command line to execute the extract. The following files will be created in the directory `/object_extracts/DDL`:

3 - Run `create_ddls.sh --version` to check the current version of the extraction scripts.

## **For Windows:**

1 - Modify `create_ddls.ps1` located in the `bin` folder.
Expand Down
11 changes: 11 additions & 0 deletions DB2/bin/create_ddls.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,15 @@
#!/bin/bash

#This version should match the README.md version. Please update this version on every change request.
VERSION="Release 2024-02-28"

export versionParam=$1

if [ "$versionParam" = "--version" ]; then
echo "You are using the $VERSION of the extraction scripts"
exit 1
fi

echo "DB2 DDL Export script"
echo "Getting list of databases"
OUTPUTDIR="../object_extracts"
Expand Down
6 changes: 4 additions & 2 deletions Hive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
This repository provides some simple scripts to help exporting your Hive code
so it can be migrated to [Snowflake](https://www.snowflake.com/) using [SnowConvert](https://docs.snowconvert.com/snowconvert/apache-hive/introduction)

## Version 1.1
## Version

Release 2021-12-03
Release 2023-02-28

## Usage

Expand All @@ -18,6 +18,8 @@ The following are the steps to execute the DDL Code Generation. They can be exec

2 - After modifying, the `exp_ddl.sh` file can be run from the command line to execute the extract. The following files will be created in the current directory under `ddl_extract`:

3 - Run `create_ddls.sh --version` to check the current version of the extraction scripts.

`./exp_ddl.sh`

## Reporting issues and feedback
Expand Down
10 changes: 10 additions & 0 deletions Hive/exp_ddl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,16 @@
#use one of the 2 CLI clients below to connect
#adjust arguments below to connect to your environment, using username,password or keytab

#This version should match the README.md version. Please update this version on every change request.
VERSION="Release 2024-02-28"

export versionParam=$1

if [ "$versionParam" = "--version" ]; then
echo "You are using the $VERSION of the extraction scripts"
exit 1
fi

HOST=localhost
PORT=10000

Expand Down
6 changes: 4 additions & 2 deletions Oracle/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This repository offers a collection of straightforward scripts designed to facil

## Version

Release 2024-02-23
Release 2024-02-28

## Prerequisites

Expand All @@ -24,7 +24,7 @@ You will need the conection string to your database. You can use a connection st

For example, if your database is hosted on AWS, you will need to use the connection format provided in this link: [AWS conection string](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToOracleInstance.SQLPlus.html). Therefore, a valid example connection string would be as follows:

`TEST_USER@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=url.amazonaws.com)(PORT=1521))(CONNECT_DATA=(SID=orcl)))`
`TEST_USER/PASSWORD@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=url.amazonaws.com)(PORT=1521))(CONNECT_DATA=(SID=orcl)))`

The link contains detailed instructions on how to connect to an Oracle instance on AWS using SQL\*Plus or SQLcl. It provides the necessary format for specifying the connection details, such as the hostname, port, and SID. By following the instructions in the link, you will be able to establish a successful connection to your Oracle database hosted on AWS.

Expand All @@ -51,6 +51,8 @@ To obtain the necessary files for executing the DLL code generation, follow thes

7. Extract the contents of the .zip file by right-clicking on it and selecting the "Extract All" or similar option. Choose a destination folder where you want to extract the files.

8. Run `create_ddls.sh --version` to check the current version of the extraction scripts.

You are now ready to proceed with executing the DLL code generation using the files found in the "bin" and "script" folders.

In the "bin" folder, you will find the bash scripts for Unix/Linux environments or the batch scripts for Windows. These scripts are designed to facilitate the DLL code generation process, ensuring compatibility across different operating systems:
Expand Down
25 changes: 24 additions & 1 deletion Oracle/bin/create_ddls.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
#!/bin/bash
#GENERAL INSTRUCTIONS: This script is used to extract object DDL from your Oracle Database. Please adjust the variables below
# to match your environment. Once completed, your extracted DDL code will be stored in the object_extracts folder.


#Version 2024-02-28: Added flag to display version. Update output text with more detailed information about the execution.

#This version should match the README.md version. Please update this version on every change request.
VERSION="Release 2024-02-28"

export versionParam=$1

if [ "$versionParam" = "--version" ]; then
echo "You are using the $VERSION of the extraction scripts"
exit 1
fi

echo "[$(date '+%Y/%m/%d %l:%M:%S%p')] Info: Execute Oracle extraction scripts: Started"

export ORACLE_SID=
export CONNECT_STRING=system/oracle
export SCRIPT_PATH=
Expand All @@ -19,13 +35,16 @@ if [ ! -e "$SQLCL_PATH" ]; then
exit 1
fi

echo "[$(date '+%Y/%m/%d %l:%M:%S%p')] Info: Step 1/4 - Creating Directories: Started"

#Path to where object extracts are written
mkdir -p $OUTPUT_PATH/object_extracts
mkdir -p $OUTPUT_PATH/object_extracts/DDL
mkdir -p $OUTPUT_PATH/object_extracts/STORAGE
touch -- "${OUTPUT_PATH}/object_extracts/DDL/.sc_extracted"

echo "[$(date '+%Y/%m/%d %l:%M:%S%p')] Info: Step 1/4 - Creating Directories: Completed"


if [ ! -e "$OUTPUT_PATH" ]; then
echo "The output path does not exist."
Expand All @@ -47,4 +66,8 @@ export EXCLUDE_CONDITION="('SYSMAN')"
# Modify this JAVA variable to asign less or more memory to the JVM
# export JAVA_TOOL_OPTIONS=-Xmx4G

$SQLCL_PATH/sql $CONNECT_STRING @$SCRIPT_PATH/create_ddls.sql $INCLUDE_OPERATOR $INCLUDE_CONDITION $EXCLUDE_OPERATOR $EXCLUDE_CONDITION $OUTPUT_PATH
echo "[$(date '+%Y/%m/%d %l:%M:%S%p')] Info: Step 2/4 - Extracting DDLs: Started"

$SQLCL_PATH/sql $CONNECT_STRING @$SCRIPT_PATH/create_ddls.sql $INCLUDE_OPERATOR $INCLUDE_CONDITION $EXCLUDE_OPERATOR $EXCLUDE_CONDITION $OUTPUT_PATH


Loading

0 comments on commit b14b74c

Please sign in to comment.