An unfortunate part of data warehousing is this: There are lots of disappointments. CEOs fund projects to empower better decision making, but because data warehouses take too long to build, the outcome all too often is a warehouse that no longer meets business requirements and is hard to extend or change after go-live. It is an uncomfortable truth for IT to accept but there are a lot of failed projects in the world of data warehousing.
“Usually, data warehouse development cycles are long — they could be 18 months,” says David Stodder, director of BI research at The Data Warehousing Institute (TDWI). “There is frustration on the business side that we are not getting value fast enough.” That is why ETL Development automation gets straight to the heart of the problem by dramatically resetting delivery expectations for the better.
It tackles head on the challenge expressed by Ryan Fenner, vice president and enterprise data solutions architect at Union Bank in San Francisco: “The reputation of data warehouse projects is ‘slow progress’ and ‘big expense,'”. In Fenner’s view, business leaders “say ‘here are my requirements,’ people go into a dark room for six months and then they come out.
Agile techniques that deliver functionality in short iterations can help in data warehousing and BI but although introduced several years ago, are “still a minority practice” compared to traditional waterfall development approaches. according to Philip Russom, head of data management research at TDWI. However, if we look to Development Automation to first of all build the code base more quickly and accurately than traditional development techniques and to enable rapid iteration as requirements change then clearly we are looking at a game-changer. Using automation doesn’t mean that you have to use the Agile Methodology; it works equally well with Waterfall development but one thing is certain, Agile or Waterfall, you are going to deliver MUCH faster to the business leading to far more ‘Wow!’s’ and far fewer responses along the lines of ‘It’s not what I wanted. What took you so long? Why did it cost so much?’.