Before the advent of Oracle GoldenGate technology, data http://download. Request PDF on ResearchGate | Expert Oracle GoldenGate | Expert Oracle GoldenGate is a hands-on guide to creating and managing complex data replication. Expert Oracle GoldenGate is a hands-on guide to creating and managing DRM -free; Included format: PDF; ebooks can be used on all reading devices.

Expert Oracle Goldengate Pdf

Language:English, Japanese, French
Genre:Children & Youth
Published (Last):07.05.2016
ePub File Size:16.45 MB
PDF File Size:18.69 MB
Distribution:Free* [*Registration needed]
Uploaded by: OBDULIA

Downloading Oracle Goldengate software from E-Delivery CHAPTER 2 replication, including an example expert-oracle-golden-gate at Grant Database Privileges and Credentials to Oracle GoldenGate Director Server Schema. Install Oracle GoldenGate Director. Installing Oracle. Oracle GoldenGate is the official name of the software. However Original Code Source: Expert Oracle GoldenGate, by: Ben Prusinski, Steve.

Explains all aspects of using GoldenGate for replication Covers homogeneous, heterogeneous, and bidirectional replication Shows the use of GoldenGate for data migration and extract, transform, load ETL. Show all. A bio is not available for this author. Table of contents 14 chapters Table of contents 14 chapters Introduction Prusinski, Ben et al. Pages Installation Prusinski, Ben et al. Architecture Prusinski, Ben et al.

Basic Replication Prusinski, Ben et al. Advanced Features Prusinski, Ben et al. Heterogeneous Replication Prusinski, Ben et al. Tuning Prusinski, Ben et al. GoldenGate Director Prusinski, Ben et al.

Disaster Recovery Replication Prusinski, Ben et al. Tips and Tricks Prusinski, Ben et al. Show next xx. By default, Goldengate stores temporary files in the dirtmp directory under the default installation directory. A good rule of thumb to use for temp file space is around 10 GB of disk space.

As a rule of thumb, we recommend that you start with at least 1 GB of disk space allocated per system for trail files. Alternatively, use the following formula that Oracle provides to determine the amount of disk space to set aside: The following query shows you how to do so: In the event of firewalls, hosts must beallowed to send and receive data for open ports that the manager, Extract, and Replicat processesrequire access to in order to send and receive data.

This range of ports must be allocated for theGoldengate environments. Also allocate ports for Goldengate manager, Extract, and Replicat processes.

By default, manageruses port We recommend that you keep this port available. In addition, keep a record of portsallocated to Goldengate processes to avoid port conflicts. You must use SP1 for these libraries. You can obtain the correct version for your specific Windows platform at www. Requirements for Microsoft Windows Clustered Environments Goldengate has some unique requirements that apply to Windows environments using Microsoft Clusters.

Execute the following steps before performing your install: As administrator privileged account, log on to one of the cluster nodes. Select a drive that has resource membership within the same Microsoft Windows Cluster group.

Make sure that the disk group has cluster node ownership. Place the Oracle Goldengate software on a shared drive accessible by all nodes in the cluster. Installing Goldengate on Windows Following is a complete run-through of a Goldengate install onto a Windows system. The process begins with downloading the software, continues through uncompressing the download file, and then moves into the install proper.

Download the Oracle 11g Goldengate software for Windows x86 platform. Create two new directories for the Oracle Goldengate software, one to be the source directory, and the other to be the target: Using Winzip or your favorite extraction utility, extract the Oracle 11g Goldengate software to the C: Execute the command ggsci on source and target: Finally, give the command EXIT to leave the ggsci environment to complete the installation process.

Do this on both the source and the target machines. Basic syntax and knowledge of copy and archive commands such as cp, mv, tar, and mkdir are essential to function within this environment. For multiple process groups with Extract and Replicat, you will need to configure the parameter files by adding SETENV as shown in the following syntax for the extract and replicat parameter files.

Following is a sample Extract parameter file: However, you will need to keep thefollowing things in mind when performing installation for Oracle Goldengate with RAC environments: You can set up the network time protocol NTP to ensure that cluster nodes are kept in sync.

Software for non-Oracle platforms can be obtained online from the E-Delivery site athttp: However, the subtle nuances lie in the preparation for these environmentswith Oracle GoldenGate. As such,Teradata requires Goldengate to perform replication activities.

Teradata communicates with Oracle 21 Oracle Goldengate functions as the replication server and receives the transactions from the Teradata CDC source server. Oracle Teradata has a number of requirements that must be performed prior to installing Oracle Goldengate. Before Oracle Goldengate can be installed for Teradata, the following items must be configured and available: For details on how to configure these features with Teradata, please consult the Teradata documentation available online at www.

The Oracle GoldenGate replication server must be configured along with setups for the source and target Teradata systems if used. Do not install on any of the Teradata Servers. Details for TAM configuration are provided in the Teradata Replication Services documentation available online at www.

In addition to the disk space storage requirements, you will need to grantoperating system—level privileges based on UNIX or Windows permissions so that the Sybase schemaaccounts and Oracle GoldenGate operating system—level accounts can read and write at the databaseand operating system level between source and target systems involved with the Sybase and OracleGoldenGate installation. Otherwise, errors will occur when the replication processes attempt to performtheir functions.

The following database requirements must be completed as part of the Sybase andOracle GoldenGate installation process: It cannot be in warm standby mode. Disk space storage requirements are the same as forgeneral Oracle GoldenGate installation, as already discussed. In addition, you will need to grant read andwrite permissions at both the database and operating system—level for the Oracle GoldenGate useraccounts on source and target DB2 systems with GoldenGate.

The following items must be configured aspart of the DB2 and GoldenGate installation: Installing Oracle GoldenGate Director 11g Oracle GoldenGate Director is an optional software module that provides an end-to-end management and administration console for Oracle GoldenGate environments. Director allows you to start and stop GoldenGate processes and to execute custom scripts as well. The installation procedure is quite a bit different for this product in contrast to that used for the base Oracle GoldenGate transaction software.

Previous releases of the Oracle GoldenGate Director software had a different architecture and software installation process. We will discuss only the current release of the Oracle GoldenGate Director software in this chapter. Further details regarding how to configure Oracle GoldenGate Director will be discussed in upcoming chapters. The installation process for Oracle 11g GoldenGate Director includes the following components: System RequirementsThe following system configurations are necessary for the Oracle 11g GoldenGate Director installation tobe performed successfully: The default port used is Next, you need to ensure that the particular platform used with Director is a supportedconfiguration.

Following are the supported platforms. Please consult the Oracle 11g Weblogic documentation for details on how to install Oracle 11g Weblogic. In addition to the above software requirements, you will also need to install a new repository database for the Oracle 11g GoldenGate Director Server during the installation process. The following browsers are supported: First you will need to create a new database schema account in therepository database on the Oracle GoldenGate Director Server.

Then you can perform the installation ofthe Oracle GoldenGate Director software.

Expert Oracle GoldenGate

The following steps must be performedto ensure a successful installation of the Oracle GoldenGate Director software: Download the Oracle 11g GoldenGate Director software. This will be used for the repository database with the Director Server.

Installing Oracle GoldenGate VeridataOracle GoldenGate Veridata performs data synchronization functionality that allows you to ensure thatall of the source and target Goldengate environments are in sync in terms of data replicated betweensource and target. Veridata uses the concept of a compare pare set of algorithms to measuresynchronization changes between source and target Goldengate environments. Veridata consists of oneor more agents and the repository server that houses the metadata objects for Oracle GoldenGateVeridata operations.

For a standard Oracle GoldenGate Veridata installation, you will need to install andconfigure the following systems: For now, we will provide you with details on the installation process. For instance, with Oracle, you must have the Oracle listener configured and online. For example, Oracle requires the following database privileges: By using a bit operating system, you will be able to take advantage of additional memoryfor virtual memory requirements should you perform large sorting operations with Veridata.

The following database platforms are supported for the Veridata Server repository: However, you will need to have staged the database repository software prior to running the installer. For instance, if you choose to use Oracle as the repository for the Veridata server, you will need to make sure that the client and server software for Oracle is available and staged on the Veridata Server host before you begin the installation process.

All of the correct values MUST be provided for the Oracle network services such as the database instance name in the tnsnames.

Database Privileges for GoldenGate Veridata Server As part of the pre-installation tasks for the Oracle GoldenGate Veridata Server, you must grant the appropriate database privileges to the Veridata schema account to be used in the installation of the Veridata Server. With Oracle as the repository database for Veridata, you need to grant database privileges to the newly created Veridata repository schema account. If you should choose to use Microsoft SQL Server for the Veridata Server repository database platform, you will need to create a new database and login for the Veridata user.

Furthermore, the following privileges will need to be granted: Thefollowing ports are used by default and should be made available: In chapter 9, we will perform an installation of the Oracle GoldenGate Veridata components. Download the Oracle 11g Goldengate Veridata software.

Unzip the Veridata software to a directory that you will use to stage the software. SummaryIn this chapter we provided you with a detailed roadmap to install the Oracle GoldenGate product familyof data replication applications. Common installation issues such as infrastructure setup for network,operating systems, and storage were provided to help you understand common pitfalls frequentlyencountered in the deployment process for Oracle GoldenGate. It also looks at some of the different replication topologies supported byGoldenGate.

Finally, you take a quick glance at some of the GoldenGate tools and utilities. You should keep some key GoldenGate architectural concepts in mind from the start: GoldenGate components can be rearranged and scaled up or downto meet specific replication scenario requirements. And GoldenGate can replicate high volumes of datain real time with low server and database impact and never miss or duplicate a transaction. Typical GoldenGate FlowMost replication scenarios involve, at a minimum, the components discussed in this chapter.

Figure shows how these components combine to form what this book refers to as the typical GoldenGate flow. The typical GoldenGate flow 33 The captured data is written to a file called the source trail. The trail is then read by a data pump, sent across the network, and written to a remote trail file by the Collector process. The delivery function reads the remote trail and updates the target database. Each of the components is managed by the Manager process.

You may notice that the data pump and the Collector process are formatted slightly differently than the others.

Each of the components serves a different purpose in the overall GoldenGate configuration. In this section, you learn about the purpose of each component and how it fits into the overall replication flow. In later chapters, you learn how to set up and configure each of the components.

GoldenGate supports a wide variety of heterogeneous source databases and platforms. Table shows the databases GoldenGate supports as source databases as of GoldenGate version 11g. In GoldenGate, the capture process is called the Extract.

In this case, the Extract is called aLocal Extract sometimes called the Primary Extract because it captures data changes from the localsource database. There are several types of extracts. You can also have aninitial-load Extract to capture database records to perform an initial load of the target database.

For example, in an Oracle database, Extract captures changes from the redologs and in some exceptional cases, the archived redo logs and writes the data to a file called the TrailFile. To reduce the amountof processing, Extract only captures committed changes and filters out other activity such as rolled-backchanges. In addition to database data manipulation language DML data, you can also capture datadefinition language DDL changes and sequences using Extract if properly configured.

You can configure Extract as a single process or multiple parallel processes depending on yourrequirements.

Each Extract process can act independently on different tables. For example, a singleExtract can capture all the changes for of the tables in a schema, or you can create multiple Extracts anddivide the tables among the Extracts.

You can stop and start eachExtract process independently. You can set up Extract to capture an entire schema using wildcarding, a single table, or a subset ofrows or columns for a table.

In addition, you can transform and filter captured data using the Extract toonly extract data meeting certain criteria. And you can generate reports automatically to show the Extract configuration. You can set these up to be updated periodically at user-defined intervals with the latest Extract processing statistics. Source Trail The Extract process sequentially writes committed transactions as they occur to a staging file that GoldenGate calls a source trail. Data is written in large blocks for high performance.

Data that is written to the trail is queued for distribution to the target server or another destination to be processed by another GoldenGate process, such as a data pump. Data in the trail files can also be encrypted by the Extract and then unencrypted by the data pump or delivery process. You can size the trail files based on the expected data volume. When the specified size is reached, a new trail file is created.

To free up disk space, you can configure GoldenGate to automatically purge trail files based on age or the total number of trail files. By default, data in the trail files is stored in a platform-independent, GoldenGate proprietary format.

In addition to the database data, each trail file contains a file header, and each record also contains its own header. Each of the GoldenGate processes keeps track of its position in the trail files using checkpoints, which are stored in separate files. For a SQL Server database, GoldenGate uses a concatenation of the virtual log-file number, a segment number within the virtual log, and the entry number.

Extract writes the CSNs to the checkpoint and trail files that you can view using the Logdump utility. If needed, you can examine the trail files in detail using the GoldenGate Logdump utility. This is sometimes necessary for debugging purposes.

You can also use Logdump to filter records and save a subset of your trail file for special processing. Although the data pump can be configured for data filtering and transformation just like the Local Extract , in many cases the data pump reads the records in the source trail and simply passes all of them on as is.

In GoldenGate terminology, this is called passthru mode. As mentioned earlier, you could set up the Local Extract to send changesdirectly from the source server to the remote target without a data pump, as shown in Figure GoldenGate flow without a data pump As you can see from Figure , in this configuration the Extract process can be directly affected bythe network. Adding a data pump to the configuration, however, introduces a layer to insulate the LocalExtract from any disruptions caused by the network connection to the target or a problem with the targetitself.

For example, if there was a network issue between the source and the target, this could cause theLocal Extract to fail. By having a data pump, the Local Extract can continue to extract changes, and onlythe data pump is impacted. This way, when the network issue is resolved, the data pump can be restartedand will quickly process the previously queued changes from the source trail that the Local Extract hasalready captured.

You can configure a single data pump or multiple data pumps, depending on the requirements. Forexample, a data pump on the source system can pump data to an intermediate or middle-tier system. The data on the middle tier can be further filtered by multiple pumps running in parallel and passed onto multiple heterogeneous targets.

In this case, no database is required on the middle tier, only the datapumps. Figure demonstrates multiple data pumps running in parallel. Data Pump 1 and DataPump 2 can pump data on to another pump or to one or more Replicats. GoldenGate flow with multiple data pumps Just as with the Local Extract, you can write any records the data pump is unable to process to a discard file for problem resolution.

Reports can be automatically generated to show the data-pump configuration and updated periodically at user-defined intervals with the latest processing statistics. The Local or data pump Extract communicates with another operating-system background Extract process called the Collector on the target. The Collector is started dynamically for each Extract process on the source that requires a network connection to the target.

The Collector listens for connections on a port configured for GoldenGate. During transmission from the source to the target, you can compress the data to reduce bandwidth. If needed, you can also encrypt the GoldenGate data sent across the network from the source and automatically decrypt it on the target. Collector The Collector process is started automatically by the Manager as needed by the Extract.

The Collector process runs in the background on the target system and writes records to the remote trail. Remote Trail The remote trail is similar to the source trail, except it is created on the remote server, which could be the target database server or some other middle tier server.

The source trails and the remote trails are stored38 They are named with a two-character prefix followedby a six-digit sequence number. The same approach for sizing for the source trail applies to the remotetrail. You should size the trail files based on the expected data volume.

When the specified size isreached, a new trail file will be created. You can also configure GoldenGate to automatically purge theremote trail files based on age or the total number of trail files to free up disk space. Just like the source trail, data in the remote trail files is stored in platform-independent,GoldenGate-proprietary format. Each remote trail file contains a file header, and each record alsocontains its own header.

The GoldenGate processes keep track of its position in the remote trail filesusing checkpoints, which are stored in separate GoldenGate files or optionally in a database table.

You can also examine the remote trail files in detail, just like the source trails, by using theGoldenGate Logdump utility. You learn more about the Logdump utility in Chapter One thing tokeep in mind is you should never manually edit and change the trail files using a text editor. Delivery Replicat Delivery is the process of applying data changes to the target database.

The Replicat applies data changes writtento the trail file by the Extract process in the same order in which they were committed on the sourcedatabase. This is done to maintain data integrity. You can configure a special Replicat to apply data toinitially load the target tables, but this is typically done using DBMS utilities such as Data Pump forOracle. Just like the Extract, you can configure Replicat as a single process or multiple parallel processesdepending on the requirements.

Each Replicat process can act independently on different tables. Forexample, a single Replicat can apply all the changes for all the tables in a schema, or you can createmultiple Replicats and the divide the tables among them. In some cases, you may need to createmultiple Replicat processes to improve performance.

Expert Oracle GoldenGate

You can stop and start each Replicat processindependently. Replicat can replicate data for an entire schema using wildcarding, a single table, or a subset of rowsor columns for a table. You can configure the Replicat to map the data from the source to the targetdatabase, transform it, and filter it to only replicate data meeting certain criteria.

Typically, filtering is done by the Extract process and not the Replicat, for performance reasons. You can write any records that Replicat is unable to process to a discard file for problem resolution. Reports can be automatically generated to show the Replicat configuration; these reports can beupdated periodically at user-defined intervals with the latest processing statistics.

GoldenGate supports a variety of target databases. Table shows the databases GoldenGate supports as target databases as of version 11g. The Manager process is the first GoldenGate process started.

The Manager then starts and stops each of the other GoldenGate processes, manages the trail files, and produces log files and reports. GoldenGate supports a wide variety of replication topologies such as one-way replication, bidirectional replication, peer-to-peer replication, broadcast replication, and integration replication.

Each of these topologies can be utilized to support many use cases. The topologies and examples of some use cases are discussed in the following sections.

One-Way Replication One-way replication, as shown in Figure , is the simplest topology and is often used for reporting or query-offloading purposes. Data is replicated from a single source to a single target in only one direction. Changes to database data are only made at the source database. The target database is read-only.

Search form

One-way replication A one-way configuration is useful for offloading intensive queries and reports from the sourcedatabase. It can also be used for maintaining a hot-standby database for failover purposes. Another concept you should keep in mind is that the source and target databases can be differentdatabase technologies. For example, the source database can be Oracle for OLTP processing, and thetarget database can contain a filtered subset of tables from the source database using the SQL Serverdatabase for reporting and analysis purposes.

A typical GoldenGate configuration to implement one-way replication is shown in Figure One-way replication GoldenGate configuration Following is a description of the process as implemented in the figure: A Local Extract running on the source server extracts data and writes to the source trail.GoldenGate is the future in replication technology from Oracle, and aims to be best-of-breed.

Data in the trail files can also be encrypted by the Extract and then unencrypted by the data pump or delivery process. Figure provides the readme instructions for the Oracle GoldenGate media pack.

In addition, you will need to grant read andwrite permissions at both the database and operating system—level for the Oracle GoldenGate useraccounts on source and target DB2 systems with GoldenGate.

By default, manageruses port Alternatively, use the following formula that Oracle provides to determine the amount of disk space to set aside:

ARDITH from Memphis
I do like reading books quaintly. Feel free to read my other posts. I have only one hobby: iceskating.