.svg)
ETL Developer
- Are you interested in business?
- Do you have some experience with coding and you’d like to learn more?
- Are you well organized?
- Do you enjoy work that requires accuracy and precision to detail?
- Do you like to ask questions but also do solitary work?
Download Facilitator Guide
CLICK HEREETL Developer could be your future career
An ETL Developer is a role within a larger family of technology roles involved with integrating systems together. Integration is a critical function within any organization and ensures that organizations work efficiently and accurately. There are a variety of types of integration roles but the most common is ETL work.
ETL stands for ‘EXTRACT, TRANSFORM, LOAD’. It’s the process of taking data from one source system and loading it into another system. While this may sound simple or boring, the ability to move data from system to system - or integrate systems together - is a mission critical function in any organization, large or small. It’s an important job for many reasons including:
- Sharing information between systems means less manual re-entry of dataLess manual re-entry means the information will be less prone to
- All organizations have data systems to support core business functions for things like
- This also means people will have the right information they need – when they need it
- And, lastly, this means that companies will have all of the information they need to make good business decisions
Let’s use health care, for example. You schedule an appointment for a doctor’s visit. When you get there the receptionist registers you and takes your biographical information. You also fill out a form for insurance. Then a nurse takes your medical history and measures your vitals. The doctor examines you and prescribes a treatment plan.
These are three different sources of data about you:
- Basic biographical information - your doctor’s office uses it to make appointments, let you know when proactive care is due, send prescriptions to your closest pharmacy, etc.
- Medical record information - private data about your health history, current health information, diagnoses and test results
- Insurance information - insurance provider or company information, membership information, coverage information and the information needed to cover the cost of doctors’ visits, medications, etc.
Each of these sources of data are entered into multiple computer systems that includes a data repository or database. These systems are often built by different companies but require the same data. At some point the relevant data needs to be extracted and shown only to the people who need it. For example, the receptionist at the doctors office does not need to see what medications you are taking - that data needs to be hidden in that system. But the pharmacist needs to see parts of your medical history to determine if you are allergic to anything. The process of ensuring that the right data is in the right seems is known as ETL - ‘EXTRACT, TRANSFORM, LOAD’.
ETL is not just for health care. Within every industry, systems need to “talk” to each other or share data. Every industry from banking and insurance to government agencies and airlines have multiple internal systems (within the company) and external systems (partner organizations) that require data.
For example, when you book a flight, the airline has systems for reservations, billing systems, passenger manifests for the airplane, systems that communicate with the Transportation Security Administration (TSA) for example. There are different types of data in this particular setting but the same function exists – to move data between the systems that require it.
The people responsible for moving data between systems are ETL Developers.
A Project Manager, who is the team leader, helps the team keep track of the project progress, and usually uses a digital tool to keep track of progress. Common digital project management tools are Excel, MS Project, Trello, Asana, Basecamp, Jira, Gantt Charts, Smartsheets, etc.
An ETL Developer is a type of software engineer that also manages the process of moving data between systems or loading the data into a central repository (known as a “data warehouse”) where data can be turned into actionable insights by analysts.
Picture a massive spider web of data flowing between systems – the ETL developer is part of the team that works to integrate it. It's the heart of the company - ensuring that the ‘blood’ and ‘oxygen’ are flowing.
Watch the first two minutes of this video
Want to get started right now?
This exercise is designed to introduce you to the type of work and thinking that is required of ETL Developers. You’ll see that this work requires paying attention and having a good eye for details. It’s a great introduction to common data structures and formats that are used in many jobs.
Try to persist through this activity. It may seem strange and awkward at first but try to figure out the concept behind the work.
You are an ETL Developer who works for a medical insurance company. The company gets information from hundreds of doctors' offices. The data comes in a spreadsheet. It needs to be loaded into the software system that the insurance company uses. Before it can be entered it needs to be in the right format.
You are an ETL Developer who works for a medical insurance company. The company gets information from hundreds of doctors' offices. The data comes in a spreadsheet. It needs to be loaded into the software system that the insurance company uses. Before it can be entered it needs to be in the right format.
CLICK ON THIS LINK TO OPEN TO AN EXCEL SPREADSHEET. IT WILL BE A VIEW ONLY FILE. GO TO THE MENU BAR AND MAKE A COPY. NOW YOU HAVE YOUR OWN COPY THAT YOU CAN WORK WITH.
Your task is to transform the data from DATA SET 1 (the source data from the doctor’s office) to DATA SET 2 (to load into the insurance company software)
DIRECTIONS:
- Follow the questions in red. There are 10 questions in all.
- Questions 3-10 require you to fill in data in the cells that have been shaded yellow. This is the work of extracting, transforming and loading up the data.
- Once you’re done you can check your answers in Tab 2. But first try it yourself!
Meet professionals in the Field
ETL Developers typically work on a data engineering, data services or data science team but their function is a bit more narrow.
ETL developers will understand the data very well and might often play an advisory role with others on the team who want to use the data.
Here are some ‘day in the life’ stories you can read about people in the data services field:
Understand the Business
Any company that works with considerable amounts of data will need the skills of an ETL Developer.
How Data Supports Organizations
- All organizations have data systems to support core business functions for things like
- Tracking worker performance and benefits such as salary raises, time off, retirement benefits - Human Resources/People Department
- Running payroll to pay workers and accounting systems to track income - Finance Department
- Inventory to keep track of supplies and ordering
- Core business and operational functions are supported by systems
- Prior to computers, most functions were performed on paper
- With the advent of computers, these systems were digitized
- The systems that support these functions need to share information or “talk” to each other
The Job Outlook
According to the Bureau of Labor Statistics the demand for Data Administrators - including ETL developers - is expected to increase by 11% through 2024. This is also a career that can springboard into many other data-related jobs and provides an excellent foundation for career growth.
There are opportunities for different skill levels and years of experience.
On average, ETL developers can earn about $73,000.
For example, junior level ETL Developers with knowledge of SQL and Talend can earn $57,000, according to HR recruiter.
- Salaries of ETL Developers fall within $33,418/year on the lower end to $166,400/year on the high end.
- On average, ETL developers can earn about $73,000.
- For example, junior level ETL Developers with knowledge of SQL and Talend can earn $57,000, according to HR recruiter.
Any job involving data warehousing is considered a high growth area.
Career Paths
ETL Developers are part of a Data Careers Path.
The IT Path
People who work with data can be part of an IT team. In a large company this team will be led by a Chief Information Officer (CIO), Chief Technology Officer (CTO) or even a Chief Data Officer.
- A common entry point is through software development or data-related coding jobs. Training and experience with SQL, Python or any open source or proprietary tool can be a first step. Note that the tools and techniques for this work are becoming more efficient. Some of the new tools are drag and drop that don’t require coding. In the future, it will become possible to move into an ETL role without much coding experience.
- Another entry point to ETL Developer roles is through Networking. These roles are accessible at an entry level through AWS, CompTIA Cloud Essentials or any Cloud Practitioner certifications. Many large tech companies such as Amazon, IBM or Google offer these for free.
- ETL Developers sometimes are Data Analysts or Business Intelligence / Data Report writers first.
- ETL Developers can later become Data Analysts or data scientists.
- ETL Developer roles can lead to roles in Data Architecture, Data Modeling or Database Administration or Business Intelligence / Report writing.
- ETL Developer can also lead to other “integration” oriented roles such as web services developers.
The Business Path
Some people who work with data are not part of a technical team, but work in business operations.
- A common entry point into a business data path is as a data entry clerk. This entry level job involves entering data via spreadsheets, like Excel, or Customer Relations Management (CRM) platforms like Salesforce.
- Analysts collect, synthesize, interpret and present the data. The Data Analyst role can have specialist function and skill. For example, Business Analysts, Financial Analysts, Marketing Analysts. There are also specialized skill sets within data analytics, such as Data Visualization or Data Quality.
- The business path can lead to advanced careers in Business Intelligence or a field of speciality, such as Data Visualization or Data Quality.
The path to becoming an ETL developer is not as straightforward as some other roles.
An ETL Developer is a good path to pursue if you have some background in coding - like Python - and you’re willing to learn tools - like Informatica and Talend, and newer software like Airbyte and Jitsu.
You can get started by learning an ETL tool on your own. These are free foundational courses for ETL Developers. For example, Khan Academy or register to try a free version of company software like Airbyte or Jitsu and play around with it to learn.
Learning online at your own pace is a great way to start. But avoid paying for a course until you are very certain that this is a career path with a job outcome for you. This article has some helpful tips.
Skills to Pay the Bills
Beginning to master some of the ETL tools is a sure way to show your motivation and start your career journey.
Skills that will help you succeed in the role of ETL developer include:
- Be passionate about data
- Have good attention to detail
- Have the motivation to master ETL tools/software
- Care about the completeness, accuracy, consistency and security of data
- Develop the practical skills required to complete actual ETL and data-related projects.
Review a Job Description
Job Descriptions are the way a company recruits and hires talent. You can learn a lot from a Job Description about the skills and qualifications you need to prepare for the job. Even if you are not ready to apply for a job now, learning to read the Job Description can help you prepare for what you need to learn after high school.
This company (name withheld) is hiring a Junior ETL Developer. This is an entry level job. Read the job description and take note of all the skills you need to apply for the job. As you read, look for the technical skills that you will need to learn, and the human behavior skills that you are already good at.
GET PREPARED
Make Your Plan
Be ready to apply to this job in a few years by writing a paragraph of less than 250 words explaining why you want the job and why you will be a good fit for the company.
- Why you want to pursue a career as an ETL Developer and specifically what aspects of the job interest you
- What skills you have that will be an asset for this kind of job
- What skills you need to learn after high school
- What is the career path you see for yourself in the future?
Keep this paragraph as part of your career plan.
