In today’s digital age, data plays a crucial role in the functioning of educational institutions. One of the key tools that schools can leverage to manage and utilize their student and financial data effectively is a data warehouse. But what exactly is a data warehouse, and how does it differ from a database? How can it help my school operations? In this blog post, I dive into these topics:
- What is data warehousing vs. a database?
- Why should my school consider a data warehouse?
- What components make up a data warehouse?
- What are the different approaches and tools?
- My real-life school use case
What is Data Warehousing?
A data warehouse is considered a core component of business intelligence. Unlike a database, which is designed to store and manage current operational data needed to power day-to-day applications, a data warehouse is a repository for current and historical data from multiple systems, specifically structured for analysis and reporting. A data warehouse allows users to identify trends and patterns through complex queries across large datasets.
In private and independent schools, implementing effective data warehousing strategies can help enhance educational outcomes.
Why Should You Consider a Data Warehouse?
There are several reasons why schools should consider implementing a data warehouse:
- Decoupling: A data warehouse allows you to decouple your data from your data source, enabling you to transform data into formats that may be more useful for the unique ways your school operates.
- Extensibility: With a data warehouse, you can use your data in ways that aren’t supported out of the box. This includes building custom tools for your school.
- Integration: Data warehousing lets you marry data from multiple sources all in one place, making it easier to conduct institutional research, administrative tasks, and IT work such as account provisioning.
- Tool Variety: You can use a wider variety of tools, including free open-source solutions. DIY can also be budget friendly.
- Resilience: Having your own data warehouse can help eliminate downtime, prepare for migrations, and provide secure backup storage.
The Layers of a Data Warehouse
A data warehouse is composed of several layers, each serving a specific function. This is how I define those layers:
- Orchestration: Managing the workflow of data processes
- Extraction: Pulling data from various sources
- Transformation: Converting data into a usable format
- Loading: Inserting data into the warehouse
- Storage: Storing the data securely
- Modeling: Structuring the data for analysis
- Business Intelligence (BI): Using tools to analyze and visualize the data
Different Approaches and Tools
There are various approaches to data warehousing, including cloud-based and on-premises solutions. Each has its own set of advantages and disadvantages:
- Cloud: Offers scalability and flexibility
- On-premises: Can be more cost effective if you already have the necessary infrastructure
When selecting tools for your data warehouse, consider the following:
- Free and Open Source: These tools are budget-friendly and have a large support community.
- Familiarity: Choose tools with which you are comfortable. In my case, it was Python and the Microsoft Power Platform.
- Compatibility: Ensure the tools are compatible with your existing systems, such as Power BI for data modeling
Data Warehousing at The Bush School
As the Database and Cloud Services Manager at The Bush School, I had to find a solution when our old transcript system, Crystal Reports, was no longer supported. We are a Blackbaud school and a Microsoft school, yet our grading methods are unique and the built-in transcripts in Blackbaud’s SIS don’t work for our purposes. The faculty aren’t going to change the way they do things, so I had to either find a company that made the kind of transcripts we needed and that integrated securely with our existing systems, or I had to build something myself.
I chose to go with an on-premises approach and was able to find free, open-source tools that had lots of users to offer support. I’m more comfortable with Python than no-code solutions and was able to leverage Power BI due to our existing Microsoft licenses. Using data stored in our warehouse, I built a custom transcript report using this process:
- Extract, transform, and load (ETL) with Python
- Orchestrate with Prefect (an open-source Python tool)
- Authenticate, make API calls, and parse data with Python
- Transform data with PostgreSQL to set up transcripts, GPA, and enrollment tables
- Model in Power BI using Power Query to pulls from these tables
- Power BI Report Builder creates the actual transcript reports
This all now runs automatically every day to provide real-time custom transcripts.
Our school’s unique grading methods require a tailored solution. By leveraging our data warehouse and building our own transcript report, we can now meet the needs of our college counselors and the registrar, ensuring everyone has the information they need at their fingertips.
If you want to dive deeper into data warehousing, check out this free webinar Blackbaud hosted with me and my colleague, Hudson Harper of The Downtown School.
