Published in Memphis.dev·2 days agoBatch Processing vs Stream Processing: Why Batch is dying and Streaming takes overIn the digital age, data is the new currency and is being used everywhere. From social media to IoT devices, businesses are generating more data than ever before. With this data comes the challenge of processing it in a timely and efficient way. Companies all over the world are investing in…Datastream14 min readDatastream14 min read
Published in Memphis.dev·Mar 22Journey to Event Driven — Part 2: Programming Models for the Event-Driven ArchitectureWhat is Event-Driven Architecture? Before delving into different programming models for Event-Driven Architecture, let’s first try to understand what exactly Event-Driven Architecture is. Event-Driven Architecture is an architecture paradigm that uses events to communicate among decoupled services. It is a well-known architecture when dealing with applications built with microservices. …Live Streaming Data20 min readLive Streaming Data20 min read
Published in Memphis.dev·Mar 13Stateful stream processing with Memphis and Apache SparkAmazon Simple Storage Service (S3) is a highly scalable, durable, and secure object storage service offered by Amazon Web Services (AWS). S3 allows businesses to store and retrieve any amount of data from anywhere on the web by making use of its enterprise-level services. S3 is designed to be highly…Memphis Dev12 min readMemphis Dev12 min read
Published in Memphis.dev·Mar 9Stateful stream processing with Memphis and Apache IcebergAmazon Web Services S3 (Simple Storage Service) is a fully managed cloud storage service designed to store and access any amount of data anywhere. It is an object-based storage system that enables data storage and retrieval while providing various features such as data security, high availability, and easy access. …Apache Iceberg10 min readApache Iceberg10 min read
Published in Memphis.dev·Mar 1Comparing Top 3 Schema Management ToolsBefore deepening into the different supporting technologies, let’s create a baseline about schemas and message brokers or async server-server communication. Schema = Struct. The shape and format of a “message” are built and delivered between different applications/services/electronic entities. Schemas can be found in SQL & No SQL databases, in different…Schemaregistry5 min readSchemaregistry5 min read
Published in Memphis.dev·Feb 16Top 8 to cover before taking your messaging queue to productionMessage brokers are very powerful components in the architecture of distributed applications, which should absorb an indefinite, unknown, or unforeseen amount of workloads and data movement. When using a message broker or a messaging queue in production, it is often the crucial component that has to be alive for the…Message Queue3 min readMessage Queue3 min read
Published in Memphis.dev·Feb 15The Future of SQL: Databases Meet Stream ProcessingSQL was created in 1974 and has become a very useful and popular tool for programmers who develop applications for various businesses. Even though this is a fairly simple programming language, it provides a lot of significant advantages, which makes it an important and necessary tool for data scientists. However…Sql22 min readSql22 min read
Published in Memphis.dev·Feb 8Streaming-First Infrastructure for Real-Time Machine LearningReal-time machine learning refers to continually improving a machine learning model by feeding it actual data. A data analyst /scientist or a developer creates the model offline using a collection of the previous testing set. All sectors and industries try their best to fetch all possible benefits from machine learning…Machine Learning15 min readMachine Learning15 min read
Published in Memphis.dev·Jan 29Supabase CDC Webhooks to Memphis REST GatewayIntroduction Supabase is an open-source Firebase alternative. In databases, change data capture (or CDC) is a set of software design patterns used to determine and track the data that has changed so that action can be taken using the changed data or because of it. In simple words: Each time a…Supabase4 min readSupabase4 min read
Published in Memphis.dev·Jan 22Connect Memphis as an Argo event sourceIntroduction Argo is a collection of open-source tools for Kubernetes to run workflows, manage clusters, and do GitOps easily. Memphis is an open-source next-generation alternative to traditional message brokers. Among Argo tools, Argo Workflows can be found. Kubernetes-native workflow engine supporting DAG and step-based workflows. Some of Argo Workflows features —…Argo4 min readArgo4 min read
