Over the course of an 8-month project involving 6,000 hours of engineering work, Foursys modernized the data architecture of a major Tier 1 financial institution, unifying 12 legacy systems into a Data Lakehouse powered by Databricks on AWS.
The transformation reduced the latency of risk models from 24 hours to just 15 minutes, gave the business team full analytical autonomy, and unlocked R$45 million in credit provision optimization within the first 6 months of the platform’s deployment.
The Challenge: Fragmented Legacy Data and Reactive Risk Decisions
Operating with strategic credit, risk, and operations data scattered across 12 distinct legacy systems without native integration created severe bottlenecks for the Tier 1 financial institution.
The lack of unification made cross-functional analyses time-consuming and required weeks of manual work for data extraction and reconciliation. As a result, decisions regarding loan provisions and credit approvals operated with a 24-hour lag, acting reactively rather than predictively.
Key Challenges:
- Operational Silos and High Data Latency: Risk models operating with outdated data, limiting the ability to react to market changes.
- IT Service Bottleneck: Completedependence on the IT team to generate business queries and reports, resulting in wait times of weeks.
- Regulatory Pressure and Capital Optimization: Requirements from the Central Bank of Brazil (BACEN) for more agile risk models and the urgent need to optimize the allocation of credit provisions.
Why Foursys?
To lead a project of this scale at a large (Tier 1) financial institution, the client sought a partner with proven capabilities in advanced data engineering and execution that would not pose a risk to operations.
Foursys was selected based on the following differentiators:
- Advanced Expertise in Data Lakehouse and Cloud: Consolidated mastery of the Databricks architecture on AWS S3 and Apache Spark for high-volume processing.
- Experience in Critical Tier 1 Banking Environments: Ability to operate under strict BACEN regulatory requirements and integrate legacy systems without operational impact.
- Technical Mobilization Capacity (6,000 hours in 8 months): Infrastructure and multidisciplinary teams capable of meeting challenging deadlines without compromising quality.
- Focus on Business Autonomy (Self-Service): A methodology geared toward empowering the bank’s teams, ensuring that the platform’s value is directly harnessed by the business units.
The Solution: Data Lakehouse Architecture on AWS with Databricks
Foursys designed and implemented a unified data platform based on the Data Lakehouse concept, adopting the Medallion Architecture (Bronze, Silver, and Gold) for the organization and governance of data flows:
- Real-Time Ingestion (Bronze Tier): Continuous connection to the 12 legacy sources via streaming and ACID transactions through Delta Lake.
- Standardization and Quality (Silver Tier): Bulk datacleansing , enrichment, and reconciliation using Apache Spark and Python.
- Risk and Credit Modeling (Gold Tier): Curated datasets to feed predictive risk and credit provisioning models, updated every 15 minutes.
- Governance and Unified Catalog: Structuring of a centralized catalog, data lineage, and compliance rules aligned with BACEN.
- Self-Service Analytics & Executive Dashboards: Integration with Power BI for management reports and technical training for the business team.
Key Technologies
- Databricks: Unified platform for data engineering and data intelligence.
- AWS S3: Scalable and secure tiered storage.
- Apache Spark: High-performance distributed processing.
- Delta Lake: ACID transaction guarantees and data history (time travel).
- Power BI: Executive visualization and analytics layer.
- Python: Language for ETL pipelines and feature engineering.
Success Factors and Overcome Challenges
- Unification of 12 Heterogeneous Sources: Harmonization of data from systems with completely different formats, frequencies, and structures.
- Ensuring Auditability and BACEN Compliance: End-to-end governance of the data pipeline for transparent validation of risk calculations.
- Cultural Transformation Toward Autonomy: Training analytics teams, eliminating dependence on IT, and decentralizing access to business data.
Results
The new data platform transformed the bank’s analytical and financial capabilities:
- R$ 45 Million in Optimization: Identification of efficiencies in the allocation of credit provisions within the first 6 months.
- Risk Models Every 15 Minutes: Reduction in model update time from 24 hours to 15 minutes.
- 12 Centralized DataSources: Elimination of data silos between the credit, risk, and operations departments.
- Independence of the Analytics Team: Elimination of dependence on IT for daily queries and report generation.