November 1996
Sales by ORACLE Partners are Growing
Treehouse Software's tRelational and DPS (ADABAS Data Propagation System) products are becoming widely known and accepted. The idea of "moving" ADABAS data has been around for a long time. Whether this movement is referred to as migration, replication, conversion, transformation or warehousing, the disciplines are similar when moving this non-relational data to any relational database.
George Szakach of Treehouse Software says, "We were first approached to do such a project in 1987 for a large bank in California. The target of the 'conversion' was DB2. Nobody talked 'data warehousing' back then. Since that time, we have talked to nearly 300 sites about their plans. The bank spent a huge amount on a prototype 'interface', then scrapped the idea because they gave up on the application which caused the requirement in the first place. Some sites have been adamant about moving completely off ADABAS in six months or less, only to find in some cases that 7 to 8 years later they are still on ADABAS AND one or more additional databases! Meanwhile, their ADABAS activity has doubled or tripled. Other sites have said they are converting or migrating without any real tools, and a few have done it, mostly 'by hand', and at great cost. And now, the tide seems to be going in the direction of warehousing."
When TSI first announced its tRelational product, significant consulting was necessary to go along with the software tools. Since then, we've improved the product to the point that sometimes no consulting is necessary. To a large extent, customers are becoming comfortable with running tRelational and DPS themselves.
tRelational handles the initial analysis required for migration of selected ADABAS files to the relational world of ORACLE, SYBASE, etc. tRelational assists in the design of the logical and physical models of the relational data by analyzing the ADABAS side. This analysis includes examining the ADABAS FDT layout of the files, incorporating valuable PREDICT data, and viewing the physical ADABAS data. Most importantly, these findings are retained in the tRelational dictionary.
By retaining all this good information, the DPS software gets a jump start in its challenging task of propagating the ADABAS changes to the various relational tables. If one ADABAS file is materialized into 10 ORACLE tables, then one update to certain fields in an ADABAS record on the file could result in anywhere from zero to ten ORACLE tables being updated, with the potential of multiple updates to each table. Customers were telling us that the handling of propagation of ADABAS changes was the most important aspect, something they would never try to do themselves.
At one DPS success site, TSI consultants assisted customer personnel in materializing an ADABAS file into ORACLE tables and setting up for propagation of changes. They took a full day's Protection Logs for two different databases, selecting the file's transactions from each of them, and adding some selection criteria. Our Log Extract module (LE) processed nearly 5 million PLOG records, extracting 5130 selected file updates, in a TCB time of .28 seconds. The Log Transformation Routine module (LTR) generated, in .01 seconds, SQL for 903 Inserts, 24 Deletes, 364 Updates, and 244 Commits. The updates were applied successfully on the ORACLE database.
Roy Parkinson, Director of the Data Propogation System Development Group at Treehouse explains, "The timings for DPS were impressive. The customer was very pleased. The software went in without difficulty, and in a short time our consultants were progressing toward a full prototype of DPS operation. From this, the customer will be able to move forward and handle more files."
Roy adds, "The tRelational and DPS software products are the result of a lot of good work by our Treehouse technicians who understand not only ADABAS, but what customers need and how to get there in a relational world. This customer has agreed to be a reference site for DPS. Please call and talk to our sales department, or to me, about your needs. Then, if appropriate, we will arrange for you to talk to this customer."
At another DPS site, we extracted 750,000 records and fed them into LTR in a total time of under 30 minutes for the extract and LTR. The CPU time was 15 seconds.
Treehouse Software has been approached by both SYBASE and PRAXIS International, Inc. to interface the DPS software with (i. e., provide an ADABAS agent for) their data replication technology. For SYBASE, this is known as the Sybase Replication Server. For PRAXIS, this is known as the PRAXIS OmniReplicator. From these data replication products, the data would be distributed to various databases and platforms.
These Replication products handle the replication of changes applied to data on a "Source" to one or more "Targets". They take care of selecting the appropriate changes, queuing them to the appropriate Target, delivering the changes, and noting success or failure of the changes to the Target. As such, the products handle a lot of the delivery issues, so it would be advantageous to interface with them.
Neither replication product handles transforming non-relational ADABAS data to a relational format. However, both replication products have interfaces which allow them to receive data from non-supported Sources. Therefore, TSI would plan to adjust or extend DPS to feed ADABAS change data to these interfaces.
There are still many things to do to DPS, such as NATURAL DATE variable conversion, ADABAS 6 support, and MU/PE empty occurrence suppression. Some sites may need variations to handle targets such as AS/400 or VAX. Meanwhile, the version 1.0 release is solid, stable, and is worthy of a look by many sites. Call Treehouse Software for details, and ask for Roy.
Return to the Table of Contents
In May of this year, we received a call from Jim Michaels of Burlington Air Express of Los Angeles. Jim stated that he needed near-immediate relief on his CPU. Average CPU utilization was 86% for 24 hours with an eight-hour average of over 96% during daily peak hours.
With the steady, predictable growth Burlington had been experiencing, things would come to a halt in 3-4 months unless some significant relief could be found, installed, quickly tested, and placed into production. A hardware upgrade would relieve the pressure, but the need would be even greater in 1997.
Jim had already tried some remedies, including Fastpath (its beta version), some other optimizing tools, and related consulting. None of these were acceptable or produced significant improvements. He was interested in learning more about RACE, a product of World Quality Systems Limited of Great Britain, marketed and supported by Treehouse Software. Being a very senior technician, Jim looked through the reference manual and was able to see the value of RACE immediately. Jim wanted to be sure that the product worked as advertised, and said that price and being an initial site in the U. S. were concerns of his management.
TSI immediately put a team together to handle this important site. Roy Parkinson led the team of technical experts to get the product to Jim, assist with any installation difficulties, and answer questions. Roy interfaced with Alan Archbold of WQSL, and we got Jim and Alan together. Rich Jacobson, Vice President of TSI, led the marketing team to ensure Jim received the proper attention and, of course, a good price. Rich interfaced with Len Jenkinson, Managing Director of WQSL, and former Managing Director of the Software AG U. K. office.
Jim employed the RACE Call Monitor, an easily-installed, unobtrusive tool, to generate statistics about what RACE would save in resources had it been in use. He was encouraged by the numbers and proceeded to install the complete RACE product.
"As with any new product, there are problems," says Roy, "but Alan and Len worked with us to quickly resolve these. The smallest problems can look very big on a production system. You can't just handle a million calls and fail on the next one. You can run RACE in a test environment, but only a production environment, preferably a very busy one, is where RACE will shine. The RACE product had to be close to foolproof, and it was."
Some initial numbers from Jim indicated that RACE would be a success. Burlington had 2691 NATURAL modules making 8720 different types of calls against 19 ADABAS files in a 2 1/2 day period. This amounted to 191 million calls to ADABAS. RACE took action on 128 million, doing the decompression rather than passing these on to ADABAS. RACE also eliminated thousands of RC commands and over 1 million redundant read calls. The RACE Prefetch (on-line and batch) and call elimination resulted in a savings of 40% in CPU consumption.
RACE has a "get smart" functionality that impressed Jim. RACE monitors every ADABAS call and keeps statistics on the Prefetch processing characteristics and dynamically adjusts the Prefetch buffers to optimize performance. Jim also appreciated that RACE can be customized down to the application, file, and/or program level.
This initial success warranted a closer look and a more detailed stress test. Remember, Jim was "testing in production".
In a recent discussion with Rich , Jim said "We couldn't wait. We upgraded to a 3090 600J last July, with a CMOS R73 looking possible in the future. Batch was the big problem back then. Before the 600J, there was a 1200 batch job backlog during Burlington's end of month processing and daily backlogs in the low hundreds. However, processing requirements would be increasing due to large seasonal business volume increases. The 'J' got the backlog down, but we needed more help."
Jim added, "We had ADABAS Prefetch turned on, but of course only for batch. We were saving millions of calls to ADABAS already. Once Burlington placed RACE, with its own Prefetch, into production, there was a further significant decrease in ADABAS calls. At one point, we measured the total reduction at 65.5%. Much of the reduction in calls is due to RACE doing its intelligent Prefetch, and preempting SAG's Prefetch. With 70% of Burlington's batch transactions occurring during their peak hours of 6:00 a. m. to 2:30 p. m., the improvement was very visible. One batch job that normally ran in 6.5 hours now runs in 3.5 hours. The most significant savings was in CPU time. ADABAS had fewer calls to process, obviously saving time, but also ADABAS was processing the remaining calls more efficiently. The buffer efficiency went from a range of about 16-22 to around 30 .
RACE normally waits for trends to develop before deciding on how and what to Prefetch. Just imagine being able to view 400 million Command Log records over several days, and coming to the conclusion "Let's set up Prefetch for these calls, but not those." RACE does this trend analysis in real-time, and acts accordingly to enliven Prefetch. In Burlington's case, a change was made to RACE to not wait, and instead to immediately have Prefetch do its optimization for batch programs. Jim was pleased that this capability was available in RACE, as it seemed to have a big impact for Burlington.
Some recent statistics Jim provided show a reduction in ADABAS commands from 397 million during one five-day period (with Prefetch, but without RACE) to 321 million (with RACE) the following week, with very similar processing. Along with savings of 76 million commands, Burlington saved about 25% (32 hours) of the ADABAS CPU time, and another 2.5 hours in NATURAL.
Jim says "Without RACE, we would be in trouble, I can't consider turning RACE off." Mitch Doricich, Treehouse Software's National Sales Manager, adds, "We wouldn't be able to market RACE as effectively without the initial efforts of talented individuals like Jim Michaels, at sites with pressing production requirements and few CPU cycles to spare."
Return to the Table of Contents
by Len Jenkinson, World Quality Systems Limited Two years ago, technical staff at WQSL carried out research to determine where the overhead of ADABAS/NATURAL applications was and how it could be reduced.
The CPU usage involved in making an ADABAS call can easily outweigh that of ADABAS itself processing the call. This is largely because of the need to overcome operating environment constraints in communicating between separate processes (i.e. the application running under the tp monitor and the ADABAS nucleus). In addition, the overhead in NATURAL of preparing to do an ADABAS call is not insignificant. An unfortunate by-product of a well-designed, modular NATURAL application is a need to reread ADABAS records for different screens or functions.
ADABAS data decompression and dynamic field extraction account for a substantial proportion of the CPU cost of processing calls. However, for most sites and many applications, identical calls, or calls to reread the same record, are made thousands or even millions of times each day. Often, this ADABAS power and flexibility becomes an overhead rather than a benefit.
We determined that we could develop a product, RACE, to address the problem, focusing on four areas where we could greatly improve performance. These are:
Pre-analyzing applications to determine the best optimization technique(s) for each ADABAS call
Implementing intelligent use of ADABAS Prefetch for on-line as well as batch processes
Improving record decompression performance, particularly for large and complex data structures
Eliminating redundant and unnecessary ADABAS calls
Together these techniques can bring dramatic reductions in the CPU cost of using ADABAS. Savings of 30% are common and the benefit can be much greater depending on the nature of the applications.
Current and planned developments will bring further significant performance improvements. These developments include:
Managing the reusability of multiple records per ADABAS file for individual users as well as across multiple users
Better RC command management to further eliminate unnecessary RCs
Implementing improved background buffer management techniques
Once we had the product completed, we knew we had to arrange for proper representation in North America. We knew that Treehouse Software is well-established and well-respected in the Software AG marketplace. They have the ADABAS/NATURAL knowledge, a complementary set of performance tools, a large customer base, and the variety of customers and environments that could help us shake out any wrinkles in our code. Alan Archbold of WQSL visited TSI to conduct technical discussions, and TSI was soon signed on to represent WQSL.
RACE is developed in the U. K. by World Quality Systems Limited and is marketed and supported in North America by Treehouse Software, Inc. Call TSI for more information or for a free copy of the RACE ADABAS Call Monitor which will show you what benefits you can get from RACE.
Return to the Table of Contents
You will soon be able to purchase these three products from TSI. To learn more about them, see the margin articles in this issue.
Return to the Table of Contents
As reported in the 1996 Premiere Issue of the ORACLE Informant, within three years, ORACLE intends to distribute 50 percent of its software through partners. For every partner dollar that ORACLE expects to generate, it predicts its partner will generate five! During the past year, ORACLE has seen an 89 percent growth in worldwide revenue through sales by partners-up from US$294M in FY'94-to an expected US$556M in FY'95.
ORACLE's Business Alliance Program (BAP), of which Treehouse Software is a member, provides marketing and sales assistance to its partners through trade shows, seminars and other special initiatives. BAP focuses on company-to-company relationships, with an emphasis on marketing issues.
TSI attended the ORACLE Conference in San Francisco in November. Attendance was 15,000! TSI had a display in the exhibition area along with about 400 other companies.
Return to the Table of Contents
Disclaimer: All product names mentioned in TREETIPS are trademarks and/or products of their respective holders. The mention of any non-TSI product in TREETIPS should not be considered to imply support or endorsement by Treehouse Software, Inc., its employees, or affiliates.
Return to the Table of Contents