Development of automated BI and Database cleanups

About project:

Client overview

A leading data analytics company sought assistance in managing their rapidly expanding Metabase instance, which had become cluttered with unused collections, reports, dashboards, and outdated permission groups. The accumulation of unnecessary data and access points was impacting system performance and user experience, making it increasingly difficult for their team to effectively utilize the platform.

Tech Stack:

Metabase, PostgreSQL 

Tech stack after migration:

Metabase, SQL, Python, Scheduled Automation Scripts with Airflow

Time to deliver project:

4-6 Weeks

Problem

  • The client was facing uncontrolled growth in the size of their Metabase, with unused collections, reports, dashboards, and obsolete permission groups accumulating over time. This clutter made it difficult to manage and navigate the system.

Inspection

  • The root cause was the lack of tools to "clean" the Metabase. Unused collections and dashboards remained because no one had access to remove them, and permission groups became obsolete as users were deactivated. Additionally, Metabase had access to certain schemas and data that were no longer in use. This unnecessary accumulation contributed to the growing size and complexity of the Metabase.

Recommendation

  • In systems prone to continuous expansion, like Metabase, it’s essential to implement automated processes for archiving or deleting unnecessary elements. Regularly removing unused objects ensures transparency, ease of use, and optimal system performance.

Resolution

We automated the cleanup process for Metabase and related data sources. The system now regularly removes unused reports, dashboards, and outdated permissions, and it also disconnects Metabase from unused data schemas. This automation has streamlined the system, reduced clutter, and improved overall performance, leading to a more efficient and manageable Metabase environment.

Similar projects

Do you want
the same one?

Leave a request and our manager will contact you to discuss your project and give an assessment of a similar project.

Please enter your name

Please enter your email

Please enter valid email

Please enter valid phone number

Our website use cookies
Read our Privacy Policy.
Order an audit

Please enter your name

Please enter your email

Please enter valid email

Please enter valid phone number

Order Black box audit

Please enter your name

Please enter your email

Please enter valid email

Please enter valid phone number

Order White box audit

Please enter your name

Please enter your email

Please enter valid email

Please enter valid phone number