Hello everyone.  RapidACE v2 has made tremendous strides forward in the past year.  It is coming along quite nicely in fact.  This post will introduce you to v2 major functions, and in a few of the new pages I will post – I will provide an overview of those new functions.

Please note: use the contact us form instead of trying to make comments, because comments are closed.

The idea and hope is: that RapidACE v2 will be made available in a SaaS format for members who are students at http://LearnDataVault.com/training classes.  Right now, we don’t have a membership area setup – just individual classes that you can purchase.  But in the future, membership will be an option, and hopefully RapidACE v2 will also be available as a service on that site.

In the mean time, you can hire me as a consultant to use RapidACE v2 on your projects, to help you speed it up.  Unfortunately RapidACE v2 cannot be purchased as a software package – because of the following reasons:

• I’m not a software shop
• I can’t support the customers who “would buy if they could buy”
• there’s not enough functionality there without me behind the code to make it worthwhile for you to purchase it
• it’s a complicated tool, requiring a ton of preparation to make it work properly

Ok, now that this is out of the way, let me also say this:

• RapidACE V2 will never be open-sourced
• RapidACE v2 is a complete re-write of the internal code
• RapidACE v2 is 70x more powerful than RapidACE v1
• YOU DO NOT need to use, or be using a Data Vault model to use RapidACE V2!!

Ok, so now the major functions…  Please note, at each step, you can edit the model and the cross-reference that are produced outside of RapidACE using your own data modeling tool set, then push the changes back in to either the same or another function.

RapidXREF

This particular piece reads in: multiple source data models, 1 target data model, and a sparsely populated cross-reference spreadsheet (if you have it), and produces or builds a brand new fully-qualified cross-reference that maps every source table to every target table in a matter of seconds.

By sparsely populated, what I mean is: you can feed it a) no xref at all, and it will do a best guess, b) business keys to HUBS only and it will figure out the rest with 89% to 98% accuracy.  This makes the job of building these dreaded cross-reference spread-sheets that much easier!

Yes, it reads xlsx formats, and writes xls formats directly.

RapidMerge

This function basically takes one or more source models including cobol copybooks and will merge them in to a single data model.  It will also allow you to set parameters which govern the merging of super and subset tables, or like structures together.  It also normalizes the copybooks formats.

This function also reads a simple ontology of business terms and abbreviations to allow it to detect naming convention differences across multiple models.  It uses this ontology of terms to connect and identify potentially similar table structures.

This function will also consolidate the data model, and apply reduction techniques and relationship folding techniques in an attempt to clean up the model.

As a result of the merge, it generates a cross-reference and a new data model which you can then take outside the tool, make modifications, and push back in to the tool – either RapidGEN, or RapidModel as a next step.

For instance, if you take a disparate set of star schema models, and push them through merge, you would end up with a “reduced” star schema model (if reduction is possible without changing grain).  Yes you can take disparate star schema models, push them through this function in an attempt to end up with an enterprise single consolidated star schema model.

By the way, IBM has a product for this, and I think (if I’m not mistaken) they charge over $100,000 USD for this product.

RapidModel

This function does what you think it does.  It forward engineers data models.  It will read in one source data model and then builds new integrated models: staging models, data vault models, star schema models, exploration mart models, and master data models.

It also produces or generates the cross-reference for you.

You can then take both the cross-reference and the data models, edit them outside RapidACE in your standard tool sets, and then push the edits back in to RapidGEN functionality.

This function no longer relies on Primary or foreign key declarations alone!  IF they are defined, it helps, but this function relies now on business key declarations along with sparsely declared cross-references to do it’s job.  It will also search for possible relationships that previously are undeclared.

RapidTest

This function produces SQL test cases based on a customizable template, that check a) the structural integrity of the table, b) the data set in the table: for instance, balancing staging tables to Data Vault tables, or raw dimensions and raw facts.

This is merely a time saver for assisting with unit testing of your ETL routines.

RapidData

This function produces very small amounts of test data for your model.  Quite frankly, this is not the best function nor the best use of RapidACE v2.  If you need LARGE scale test data, you should really look at the RowGen tool from CoSort (I.R.I. and company), that tool is phenomenal.

This tool is limited, but will produce up to 10,000 rows per table for testing.  It’s especially helpful for “loading the staging area”, then running the unit tests generated by RapidTest after running the ETL to load the Data Vault.

RapidGEN

This function is what you’d expect it to be, yes, it’s a code generator. This function reads source data model, target data model, a cross-reference, job files, configuration files, and a template, and then produces code.  Yes, this is template driven.  No the templates are not public today.  Heck, the software isn’t even available except through me.

Today it generates loading code from relational source to relational staging areas, and from relational staging areas to Data Vault structures.  In the future it will produce additional code for staging areas to Star Schemas, and Data Vault to Star Schemas as well.

Today, it does produce Informatica v9.1 XML files with shared sources, targets, transformations – and shortcuts, mappings, maplets, sessions, worklets, and workflows ready to run.

Today it does produce stored procedure code for loading staging areas to a Data Vault for SQLServer 2008 R2.

In the future it will produce other things, like SSIS DTSX packages, and Pentaho Kettle XML and so on.

Conclusion:

Well that’s it folks, I hope you enjoyed this overview.  Soon I will dive a bit deeper and show you the “process workflow”, as well as some diagrams that explain the working pieces.  In the future I also hope to post a few videos – BUT rapidACE v2 has no GUI it is command line driven for now, so that it’s easier to run it on the web as a service.