Nowadays, organizations need to migrate and move data from the Oracle database to various data lakes, such as Microsoft Fabric and Snowflake.
These data lakes are used in data analytics and by data scientists which help organizations to process vast amounts of data in real time or batch, derive insights, and make data-driven decisions.
Oracle GoldenGate is a very powerful tool to capture and replicate data, followed by converting it into various format, e.g., Json/parquet.
Let’s dive into using Oracle GoldenGate for the extraction of data, configuring the replication process for distributed applications/analytics, and then converting data into Json/parquet format.
Environment Detail (Source): RHEL(8.10) || 19c (19.25) Oracle Database (GoldenGate binaries for Oracle (extract process) and GoldenGate binaries for distributed applications/analytics (Replicate process)
Environment Detail (Target): Azure Cloud || ADLS || Microsoft Azure BLOB storage
(Step 1) Download GoldenGate Binaries for Linux and for Distributed Applications and Analytics
Oracle GoldenGate Downloads
(Step 2) Install GoldenGate Binaries for Linux and for Distributed Applications and Analytics
/u01/app/oracle/product/GG/23_db || GG Binaries for Linux
/u01/app/oracle/product/GG/23_az || GG Binaries for Distributed Applications and Analytics
(Step 3) Apply Patches to the GoldenGate Binaries to Be on the Latest Version
Applied patch#37071319 using command “opatch apply” on GG Binaries for Linux(/u01/app/oracle/product/GG/23_db)
Applied patch#37108354 using command “opatch apply” on GG Binaries for Distributed Applications and Analytics (/u01/app/oracle/product/GG/23_az)
(Step 4) Configure Az Tool to Test Connectivity from On-premises to Azure Portal for Storage Account
uname -a cat /etc/redhat-release
sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc sudo dnf install -y https://packages.microsoft.com/config/rhel/8/packages-microsoftprod.rpm
sudo dnf install azure-cli dnf list --showduplicates azure-cli sudo dnf install azure-cli-2.67.0-1.el8 az version
az login --user ********* --password *********
az storage blob list --account-name ********** --container-name ******** --output table -account-key *********************
(Step 5) Perform Prep Work on Oracle Database
Enable archivelog || alter database archivelog;
Enable Force logging || alter database force logging;
Enabled Supplemental Logging || alter database add supplemental log data;
Enable Golden Gate replication || alter system set enable_goldengate_replication=true;
Create GG user || create user oggadmin identified by "*********"; Grant required permission || exec
dbms_goldengate_auth.grant_admin_privilege('oggadmin');
(Step 6) Configure Golden Gate 23c Service Manager and Create a Deployment /u01/app/oracle/product/GG/23_db/bin/oggca.sh
