This project serves as a telecom case study, showcasing the implementation of an ETL (Extract, Transform, Load) process. Using SQL Server Integration Services (SSIS). The processed data is then loaded into a Data Warehouse (DWH) designed and created using SQL Server Management Studio. This project demonstrates the end-to-end ETL pipeline for telecom data, ensuring data quality and error handeling.

Signalling data describes the subscriber network activities and interaction. Subscriber does an event such as call, rowsing, latching, SMS, etc. The network vendor collects the subscriber data in real-time and bulk every 5 min activity in one fill and push it to landing servers.

Created by Moustafa Mahmoud

The dataset is generated using a script and it does not describe the full telecom dataset for signalling but it simulates the general idea.

This section describes the input data and its format.

The required processing must be completed on this data before storing it in the data warehouse. The processing steps are as follows:

Rejected records will be stored in a separate table, and the original CSV file name will be recorded.

During the data recording process in the database, there is some additional information that needs to be recorded in the auditing table to ensure the quality of the data storage:

• The number of records in the CSV file.
• The number of records successfully stored in the database.
• The number of rejected records due to not meeting the required conditions.
• Linking the data stored in the database with the original CSV file.

After completing the data storage process in the database according to the aforementioned conditions, Archive the CSV files to another folder.

Project Phases:

1. Extract Data from CSV files.
2. Process and transfer data.
3. Deal with data that does not meet the quality requirements, such as Nulls and missing values.
4. Error Handling.
5. Archive files that have been processed and transferred.
6. Auditing the processed data.

In this section, you can find screenshots of the different stages of the ETL process, showcasing key stages, including data flow, successful control flow execution, and data loading into the Data Warehouse (DWH).