ggk-quote

Let's Connect

ggk-quote

Let's Connect

ggk-contact

+91 1234 44 4444

Data Management

Enterprise Data Management on Hadoop

Designed and implemented a suitable Hadoop architecture post analysis of various data sources

Challenges

  • The client wanted to scale up the legacy data warehouse system built on RDBMS based MPP system and perform real-time analytics on the marketing data in order to generate useful business insights
  • The client was receiving data from various data sources like SAP ERP, JDA, IRI, Acousta and many others, and was facing challenges in integrating the data from these data sources & performing real time analytics on the given data
  • The client wanted to switch to a scalable EDM system which :
    • Is easy to integrate with the other enterprise applications
    • Minimizes the effort spend on data migration and processing

Solutions

  • Analyzed the various data sources and targets and designed and implemented a suitable Hadoop architecture for the same
  • Designed near real-time ingestion system using Attunity and Kafka to perform real-time batch processing
  • Designed a data warehouse on Hive and HBase for all transactional data processing
  • Used Talend ETL tool to design and develop Spark Streaming and batch jobs
  • Designed OLAP cubes for reporting

Tools & Technologies

Hadoop, Talend, SAP Hana, Kafta, Hortonworks, Hive

Key benefits

  • Production rate can now be matched with order rate, thereby enabling faster and more informed decisions for sales, pricing, product, and manufacturing teams
  • Provides real-time operational visibility
  • Able to take quicker and informed decisions
  • Lower data integration cost
  • Easily scalable architecture
  • Ability to drill across cubes/data subject in pre-defined and ad-hoc ways
  • Attunity replicates the integrated data into Kafta streams thereby providing a comprehensive view of it to the managers
Key Benefits - Data Management - Food Industry