Background
BrightWind, a specialist in wind resource assessment, focuses on estimating future energy production for wind farms. BrightWind’s team of analysts utilises on-site data collected primarily through met masts, which are equipped with various sensors to measure wind speed, wind direction, air temperature, air pressure, and humidity. The data generated by these sensors is consolidated, creating daily data files that encompass all sensor information for a given day.
fourTheorem came in and helped us review our current architecture and come up with great solutions to solve its shortcomings – and all of this was done remotely. We look forward to working with them as our mentors as we implement their solutions and architecture.
The Challenge
As BrightWind expanded its operations, a significant portion of its analyst’s time was dedicated to manually extracting and aggregating data from daily files. To improve accuracy and reduce this data management burden, BrightWind decided to create a central platform called BrightHub.
The initial plan was to deploy and scale the platform on AWS using Kubernetes. However, issues emerged due to the desire to minimise costs by scaling down during inactive periods, resulting in extended cold start times in Kubernetes or AWS Fargate, as well as a steep learning curve in Kubernetes for new developers
The Solution
BrightWind, a specialist in wind resource assessment, focuses on estimating future energy production for wind farms. BrightWind’s team of analysts utilises on-site data collected primarily through met masts, which are equipped with various sensors to measure wind speed, wind direction, air temperature, air pressure, and humidity. The data generated by these sensors is consolidated, creating daily data files that encompass all sensor information for a given day.
The initial plan was to deploy and scale the platform on AWS using Kubernetes. However, issues emerged due to the desire to minimise costs by scaling down during inactive periods, resulting in extended cold start times in Kubernetes or AWS Fargate, as well as a steep learning curve in Kubernetes for new developers
The Outcome
Scalability
Serverless components such as S3, Lambda, and API Gateway scaled dynamically, resolving performance bottlenecks and ensuring swift, reliable management of a high volume of files.
Faster Processing
The platform’s capacity to scale efficiently led to a marked improvement in processing speed, reducing data analysis time to a matter of minutes.
Cost Reduction
Serverless services are charged based on actual usage, encompassing factors like storage, API calls, and Lambda function runs. This approach significantly reduced operational costs during periods of inactivity.
Swift Setup
Serverless offerings, managed by AWS, negate the need for manual provisioning, server management, application deployments, and other administrative tasks. The utilisation of CloudWatch simplified logging and debugging while diminishing manual management efforts, allowing BrightWind to concentrate more on platform development.
Streamlined Management
Serverless offerings, managed by AWS, negate the need for manual provisioning, server management, application deployments, and other administrative tasks. The utilisation of CloudWatch, an AWS monitoring service, simplified logging and debugging while diminishing manual management efforts, allowing BrightWind to concentrate more on platform development.