Challenges
- Migrate 50 years of satellite image data & processing to a cloud native solution at a public cloud service provider
- Cloud expertise and experience modernizing, refactoring very large on-premise solution stack to a cloud native alternative
- Training and hands-on experience working in deep cloud native environments
- Database modernization and refactoring into distributed data model
- Containerization of legacy C code processing kernels
Solutions
- Fully public cloud hosted solution that processes Landsat data from raw radio signals to finished TIFF images made available worldwide
- Migrated image processing/rendering software to a Docker containers & refactored code for standard, uniform, libraries, RPMs & build options
- Migrated 11+ million scene & data artifacts from on-premise to cloud based data lake
- Processed 117,000 scenes from satellite systems from raw data to finished images on cloud native solution
- Demonstrated same code containers running both as cloud native instances & at on an on-site container cluster
- Migrated & refactored 80TB OLTP database from on-premise instance to a combination of a data lake, NoSQL, SQLite, data warehouse & OLTP
- Cloud native CI/CD pipeline for all build, test & deployment tasks
Tools & Technologies
AWS Batch Lambda, ElastiCache, CloudTrail, Cloudwatch, ECS, Amazon S3, Redshift, Amazon SQS & SNS, Anthena, DynamoDB, Oracle, Ansible, Github, Jenkins, Docker
Key benefits
- Improved image processing performance by 100x than on-premise
- Made image processing 2,000X cheaper in the cloud than on-premise – $100.00/scene on-premise vs $0.05/scene in the cloud
- Cloud hosted solution provided a 200,000:1 price-performance advantage over doing the same work on premise
