Treehouse Software helps mainframe customers quickly take advantage of Amazon Redshift via Kafka and S3

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Treehouse Software’s customer base of enterprises using mainframes for storing mission-critical data are asking for ways to take advantage of the most advanced Cloud-based data warehouses. Recently, one of our customers sent us an inquiry asking about how tcVISION can be used to move some of their data to Amazon Redshift, the fully-managed data warehouse from AWS, for reporting and analytics using their existing Business Intelligence (BI) tools. They also expressed concerned about scalability of the data warehouse, since they may increase their data movement quite quickly in the near future.

Fortunately, one of our Cloud experts has been testing out the fastest and cleanest ways to use tcVISION to pump z/OS mainframe data into Amazon Redshift.  His testing proved that tcVISION fully supports an AWS best practices framework of: Amazon MSK(Kafka)-to-Amazon S3-to-Amazon Redshift. To achieve this within the framework, tcVISION is used to convert mainframe-based data (Adabas, VSAM, Db2, etc.) to JSON format and publish it to Kafka topics (Amazon MSK), then ultimately to Amazon Redshift as seen here…

____01_Amazon_Redshift_tcVISION

About Amazon Redshift… 


Your journey to Amazon Redshift begins today with Treehouse Software and tcVISION…

____Treehouse_AWS_Badges

Treehouse Software is an AWS Technology Partner and tcVISION is a Validated AWS Qualified Software that helps customers replicate their mainframe data between a vast array of source databases and Cloud technologies, including Amazon S3, Amazon MSK (Kafka), Amazon RDS, Amazon Redshift, DynamoDB, and many more. 

tcVISION focuses on changed data capture (CDC) when transferring information between mainframe data sources AWS. Through an innovative technology, changes occurring in any mainframe application data are tracked and captured, and then published to AWS-based data stores. From there, the door is open for the enterprise to take advantage of the latest and most popular AWS AI and ML resources.

Fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for an online tcVISION demonstration.


__AWS_On_White

Further reading: tcVISION is featured on the AWS Partner Network Blog…

The AWS Partner blog talks about tcVISION’s Mainframe-to-AWS data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe data on AWS. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.

Considerations for Planning Bi-Directional Mainframe Data Replication with tcVISION

by Joseph Brady, Director of Business Development and Cloud Alliance Leader at Treehouse Software, Inc.

Data_Modrnization

Many medium-to-large size enterprises use mainframe systems that are housing vast amounts of mission-critical data encompassing historical, customer, logistics, etc. information.  Each mainframe site is unique and can have decades worth of customizations requiring innovative approaches to establishing data replication on Cloud and open systems platforms. Fortunately for these customers, Treehouse Software has been in the mainframe software market since 1982, bringing deep experience in mainframe, Cloud, and open systems technologies, as well as delivering the tcVISION mainframe data replication product. Today, Treehouse Software is helping many enterprise mainframe customers accelerate digital transformation and successfully leverage Hybrid Cloud initiatives on the IBM Z platform, storing sensitive data on a private Cloud or local data center and simultaneously leveraging leading technologies on a managed public Cloud.

Treehouse Software’s tcVISION solution focuses on changed data capture (CDC) when transferring information between mainframe data sources and Cloud and open systems-based databases and applications. Changes occurring in the mainframe application data are then tracked and captured, and published to a variety of targets. Additionally, tcVISION supports bi-directional data replication, where changes on either platform are reflected on the other platform (e.g., a change to a PostgreSQL table in the Cloud is reflected back on mainframe), allowing the customer to modernize their application on the Cloud or open systems without disrupting the existing critical work on the legacy system. tcVISION’s bi-directional replication writes directly to the mainframe database, thereby bypassing all mainframe business logic, so this architecture requires careful planning, as well as thorough and repeated testing.

Plan carefully…

The following section offers some real-world customer examples, as well as considerations and recommendations when planning bi-directional replication for any mainframe/RDBMS environments. Bi-directional replication by its nature is a very complicated undertaking, so it is necessary that customers are fully educated in all environments, software, and processes before attempting to write data back to a mainframe database. It is always recommended that customers use a minimally effective measure of bi-directional replication required to accomplish their goal — and no more. An overblown project with unnecessary bi-directional data replication invites undue complexity and delays.

Real-world customer examples…

Treehouse Software has many customers performing bi-directional data replication, and each scenario is vastly different from the others, even if some have the same sources and targets as each other.  For example, some customers utilize a Master/Master, collision-heavy proposition, while others use uni-directional one way, then “flip a switch” uni-directional the other way. Another example is a customer who has a “grand circle,” where data hits multiple applications before it finally makes its way back to an RDBMS staging database that tcVISION replicates to the mainframe.

Example of a Treehouse customer’s bi-directional data replication environment using tcVISION:

tcVISION_Adabas_To_AWS_RDS

There are many planning and implementation stages that go into a successful mainframe replication environment, and performance testing is a vital part of a successful project.  For example, customers should do performance tests on how long it takes tcVISION to read a database log, transfer data, process data, etc.  During testing at one of our reference customer sites we found a significant difference in how long it took for their test and prod LPARs to transmit data to the Cloud, based on whether the mainframe TCP/IP stack used a 32-bit or 128-bit setting.

At another site, where we are helping a large government agency perform bi-directional replication on mainframe data, their original goal was for a significant percentage of mainframe objects to have bi-directional replication. It was determined that it would be impossible to extract business logic from the existing mainframe application for usage in the downstream application. Therefore, they have decided to use a middleware product to perform the “write-back” to the mainframe database.  Given the complexity of the mainframe application, this has proven the safest way for them to proceed.

Because of the variety of customer scenarios as described above, before any site can attempt bi-directional data replication, it is crucial that they have a well-tested uni-directional process with operational controls in place for a significant time period.  “Operational controls” means processes to restart scripts, evaluation of failed transactions, orchestration of mainframe/non-mainframe DBMS changes, etc.

Please contact Treehouse Software to discuss your Mainframe-to-Cloud and Open Systems modernization plans. We can help put in place a roadmap to modernization success.


__TSI_LOGO

Contact Treehouse Software Today for a tcVISION Demo…

No matter where you want your mainframe data to go – the Cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

_0_Treehouse_tcV_Cloud_OpenSystems

Just fill out the Treehouse Software tcVISION Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.


Treehouse Software helps Mainframe Customers Replicate Data on the World’s Fastest Cloud Data Warehouse — Amazon Redshift

by Joseph Brady, Director of Business Development / AWS and Cloud Alliance Lead at Treehouse Software, Inc.

Treehouse Software is an AWS Technology Partner, and has been a mainframe software company since 1982, and in the data replication and migration market space since the mid-1990s. Today, Treehouse Software is helping enterprise mainframe customers successfully move their data to the Cloud in order to utilize all of the most advanced Cloud services and tools available. Our tcVISION product can help customers replicate their mainframe data, in real-time and bi-directionally, between a vast array of source databases and many AWS Cloud technologies, including Amazon Redshift, the world’s fastest cloud data warehouse today.

Highly concurrent workloads? Not a problem. Redshift can handle virtually unlimited concurrency. Amazon Redshift’s hybrid architecture enables unmatched performance…

___AWS_RedShift_Architecture

Shared storage provides the ability to scale to unlimited concurrency, while Redshift’s on-instance storage provides low latency access to data that can’t be achieved any other way. The unique combination of both strategies provides Redshift’s best-in-class performance today and leaves room for continued performance improvements tomorrow.

Scale quickly and automatically to handle unpredictable workloads…

Start small at $0.25 per hour and scale up to terabytes or petabytes for under $1,000 per terabyte per year. Pay only for what you use and know how much you’ll spend with predictable monthly costs. That’s 75% less expensive than the #2 cloud data warehouse provider.


Further reading: tcVISION Mainframe-to-AWS data replication is featured on the AWS Partner Network Blog…

___tcVISION_AWS_HA_Architecture

AWS recently published a blog about tcVISION’s Mainframe-to-AWS data replication capabilities, including a technical overview, security, high availability, scalability, and a step-by-step example of the creation of tcVISION metadata and scripts for replicating mainframe Db2 z/OS data to Amazon Aurora. Read the blog here: AWS Partner Network (APN) Blog: Real-Time Mainframe Data Replication to AWS with tcVISION from Treehouse Software.


__tsi_logo_400x200

Contact Treehouse Software for a Demo Today…

No matter where you want your mainframe data to go – the cloud, open systems, or any LUW target – tcVISION from Treehouse Software is your answer.

tcVISION_Overall_Diagram

Just fill out the Treehouse Software Product Demonstration Request Form and a Treehouse representative will contact you to set up a time for your online tcVISION demonstration.