Quality is delighting customers
ETL basically means extract transform and load. In a typical ETL system , data from various source systems is collected processed (transformed) and loaded in data warehouse after applying appropriate business rules.
While ETL testing, Main focus is on data. We need to verify, if data is correctly processed, all buisness rules ,caluclations are correctly applied and expected data is loaded into warehouse.
Typical techniques to achieve that are :
1. Count validation - use hash totals, control totals to verify if all rows are correctly loaded. No data loss has taken place.
2. Comparing data among st source and target - To ensure that all rules have been applied correctly.
3. Schema review/validation - To ensure if schema implemented is correct as per data model design.
4. Validating constraints - Such as unique constraints , integrity constrains have been correctly applied wherever needed.
5. Analyzing data quality - analyse data if it confirms to specified requirements , or check for any bad data.
Typical chceks include empty value checks , key value pair analysis, validating business rules etc.
6. Error handling- How your ETL system is handling rejected records.
7. Monitor overall time for package execution. etc.
good ans Shruti :)
Thanks Shruti ..
Really Nice document can u share me any materials on etl testing.
ETL testing is an extremely important process in the project development and maintenance. This is something that tend to be neglected and cause in completeness of data and lost of credibility by the end users.
I would advise you to have a look at my post to learn the best practice of ETL testing.
In this post I provide information about the basic test and validation that need to be done, plus examples from real life of cases causing discrepancy of data.
I hope you will find it useful :)
Thanks Uzi ..
I will go through the post.
The link "http://quality-gates.com/?p=1284" is not working for me, don't know why :(
It is throwing 504 Gateway Timeout ..
Please share any materials/best practices/examples etc regarding etl testing with me. My id is:
Thanks for all of your support.
Hi Ajay, It is working fine for me, and for several places around the globe.
Can you please try again.
If not try to go to: http://www.quality-gates.com/ and under News and Events, search for this post.
I also sent you by email more details.
Good luck !
Sorry but the link is not opening .. :( . i tried in mozilla firefox and in internet explorer. ..