focussing on data, energy and technology
Even though there are energy shortages in many regions of the world, a lot of energy is still being wasted. Follow a 4-step structured approach to reduce energy costs and consumption by evaluating end-use, distribution, technology replacement, and renewable energy sources.
More data stacks are shifting towards microservices and communication between them are essential as it sometimes need to communicate to more than one service. Message brokers play a critical part in this architecture by allowing services to communicate by routing messages and storing them. In this post we will be creating a real-time log viewer that consolidates errors across several apps and a python app to process error logs.
Many systems exist today that were simply not optimised for time-series data analysis and machine learning. People may not even know the data is being collected regularly as part of day-to-day operations. This is often referred to as dark data. As the hunger for data increases, data engineers are tasked to make this dark data available.