Data Engineering
Building an ETL pipeline from device to cloud (part 3)
We refactored code for our web server and created a utils module. The utils module handles file saving and ETL methods as background tasks.
blog
Data Engineering
Building an ETL pipeline from device to cloud (part 2)
In this part, I will rebuild the secure server from part 1 using FastAPI. This allows for multiple workers and running background tasks that we will be using in part 3.
Data Engineering
Building an ETL pipeline from device to cloud (part 1)
In this first part, we will be building a simple web server using the standard Python library. This is for initial testing and we will secure the web server to use HTTPS.
Data Science
Making better cheese with IoT and ML
Optimise the cheese making process utilising low-cost IoT and machine learning solutions. A small cheese factory wanted to improve its production process, focussing on the quality of its product. The key focus was to improve the temperature monitoring in their cold rooms to keep the product fresh and increase shelf-life.
Data Science
Measuring process improvements with Advanced Process Control
APC is an additional control layer on top of basic single input and single output control. APC can have multiple inputs and outputs that it evaluates together and combines this with the capability to do short term predictions in real-time. Adding the APC layer requires great expertise and adds complexity to your process and so, what are the real-world benefits?