7 Deadly Sins That Destroy A Teradata Data Warehouse
To exemplify the impact of mistakes in Teradata Data Warehouse projects, consider the analogy of a medical team. Imagine yourself as the project, preparing for a crucial and costly procedure. Naturally, you wouldn't want to hear the staff engage in the following conversations before administering the anesthesia.
1. Not knowing or losing Sight of the Strategic Business Objectives
"Does anyone know why we operate on the patient? Well, let's start anyway …"
Want more practical data engineering analysis like this?
Join DWHPro Letters and get field-tested notes on Teradata, Snowflake, AI, migrations, performance, and enterprise data work. DWHPro Letters is free. Subscribe to get new issues by email.
Most Data Warehouse projects start with a lot of enthusiasm but fail in the end because they did not meet the business objectives. Very well-designed Data Warehouses will fail if the content doesn't help to solve any business needs. Even if you see yourself as a technician first or are in a position without contact with users of the final product, never forget that whatever you produce, the customer must be satisfied.
2. Forgetting to set up Measures to evaluate the Success
"Anyone feels like this treatment is successful so far? Can we find out? No ideas? "(silence).
Even if you know your business objectives, this alone does not guarantee success. Every clear business objective is measurable. If you can't quantify, you will have a tough time commenting on your expenses and show up on top of the blacklist as soon as the subsequent restructuring occurs. The latter usually revolves in cycles of about 3 to 5 years in length.
3. Running a Teradata Data Warehouse Project like a Mass Production via an Assembly-Line
"We need one nurse for every swab and one class-2 middle belly doctor for the first opening cut. Then, the inhalation support technicians, please stand to the right and the exhalation support professionals to the left. Is our scalpel-holding team ready?"
Overspecialization has become an enormous problem in today's Data Warehouse projects. Industries with a more extended history are aware of the adverse effects of the extreme division of labor (look up "Taylorism"). Think of the Kaizen idea that became famous in the 80s and 90s - it seems to be the case that Data Warehousing is about to repeat history.
Any environment where each project member has to contribute to a very narrow range of skills, lacks understanding of the big picture, or is not expected to develop is predestined for failure.
It is very fashionable these days to create ever more exact roles to cut costs - at least at the beginning of a project. From my experience, this never actually pays off.
It just moves the costs from the beginning to the end of the project while they multiply along the way, and it is when you have to hire actual data warehouse experts to get back on track again. Be wary of so-called experts without real-world, hands-on experience.
4. Aiming for a Data Warehouse but achieving an Operational Data Store
"Let us just continue with aspirin and a little massage."
If you work with enterprise data platforms, migrations, performance tuning, or AI-driven delivery teams, DWHPro Letters is written for you. Get the next issue by email.
While it seems tempting, from a cost perspective, to pick up the source data and provide it without any costly transformations to the customer - even developers with no or only a little knowledge about data warehousing can be employed to do this, once again, this is the recipe for failure. I cannot overemphasize how important the implementation of a theoretically sound data model is.
Don't even think about leaving out surrogate keys, taking over the source system-specific naming conventions, or trying to solve the issues caused by the mentioned ODS approach in the next place, like in the data marts or reports.
5. Omitting important Roles in the Teradata Data Warehouse Setup
"Why not leave out the surgeon and the anesthesiologist and ask the ambulance driver and the caretaker to help the nurse a little?"
The usual approach nowadays seems to be to leave out essential roles from the data warehouse. Never forget: Data Warehousing is a process, not a sequence of stand-alone projects. Positions that are regularly cut include:
Data Warehouse Architect, Data Modeler, Performance Specialists, or any system-specific technical experts (Teradata experts).
The typical project team consists of some highly specialized database developers missing the big picture and a project manager without academic and hands-on experience in Data Warehousing. This shortage often accompanies a lack of good relations with business departments or even fundamental project management skills. Without the support of a technical lead (Teradata expert), which can take over the roles of the architect and the general modeling of the solution, project delays, and failure are most likely.
6. Not understanding the Target Architecture and involved Tools
"Why should there be any complication? Just slice, rip, and sew!"
We should never forget that data warehousing is a very technical topic at the end of the day. The technical component is all too often considered a low-prestige subsidiary location.
The solid education of the people involved is required at many levels. In-depth knowledge and fluency are needed in Teradata, ETL-Tools (Data Stage, Informatica, Ab Initio), and Reporting Tools (Cognos, Business Objects, Microstrategy).
Too often, these different technical components' interaction is not getting enough attention, and performance is almost always entirely ignored at the beginning of the project. Usually, omitted clarification of professional interaction and compatibility of various software can't be handled anymore at a later time. One ends up in a minefield of workarounds upon workarounds and never-ending issues around new releases that soon start to lead a life of their own.
7. Budgetary Dishonesty and an Atmosphere of Dissimulation
"What do you mean by no vital signs? It says "operation successful" on my slide! ""
The first mile on the highway to disaster always looks perfect. It appears to be brand-new. There is no speed limit, all traffic lights are green, and you seem to drive a racing car. This part of the journey ends after the first turn.
First, suppose you start a project with amounts, assumptions, or milestones that are illusory or grossly out of touch with the average experience of professionals. In that case, it does not matter if, but only when, you see yourself in unpleasant meetings, bending over backward to add reality to the plan.
Second, if the company shows a distaste for straightforwardness, clarity, transparency, and decisiveness, not only will the real problems multiply, but alongside them, the bureaucratic dissimulation tactics will take over more and more of the remaining resources.
Your best project members are usually the first to exit this race to the bottom for better opportunities.
Planning or surviving an enterprise data platform migration?
I write regularly about the performance, cost, architecture, and project mistakes that show up in real Teradata, Snowflake, Databricks, and enterprise data work.
Subscribe for free and keep launch access.
Written by Roland Wenzlofsky, founder of DWHPro and author of Teradata Query Performance Tuning. DWHPro has helped data warehouse practitioners for 15+ years.