| Welcome to the InfoQ Software Architects’ Newsletter! Each month, we bring you essential news and experience from industry peers on emerging patterns and technologies. This month, we present the highlights from the past year of the newsletter, covering topics such as cloud computing, event-driven architecture, and microservices. News BBC Online Uses Serverless to Scale Extremely Fast In a series of blog posts, Johnathan Ishmael, lead technical architect at BBC, explained why BBC Online uses serverless and how they optimize for it. According to Ishmael, BBC Online uses AWS Lambda for most of its core implementation due to its ability to scale extremely fast. The BBC website can reach 60 million browsers a day, with users requesting up to 20k pages per second. When a breaking news story erupts, traffic can increase 3x in a single minute and then keep rising after that. Matthew Clark presented more details about this architecture in a QCon Plus talk, the recording of which can be found at "BBC Online: Architecting for Scale with the Cloud and Serverless", and also an InfoQ podcast: "Matthew Clark on the BBC’s Migration from LAMP to the Cloud with AWS Lambda, React and CI/CD". Kubernetes Is Not Your Platform, It’s Just the Foundation In this article, Manuel Pais, InfoQ editor and co-author of Team Topologies, discussed that Kubernetes itself is not a platform: it is only the foundational element of an ecosystem, not only of tools and services, but also offering support as part of a compelling internal product. Platform teams should provide useful abstractions of Kubernetes complexities to reduce cognitive load on stream teams. A team-focused Kubernetes adoption requires an assessment of cognitive load and tradeoffs, clear platform and service definitions, and defined team interactions. Pais also took part in a related Q&A, "Manuel Pais on Team Topologies during COVID-19", with fellow InfoQ editor, Raf Gemmail. Matthew Skelton and Pais also presented at QCon Plus in November, and a recording of the talk can be found on InfoQ: "Business and Technical Agility with Team Topologies". Software Architecture and Design InfoQ Trends Report—April 2021 Each year the InfoQ editors discuss the current state of software architecture and design and identify the key trends to watch. This year the editors were joined by Holly Cummins, an innovation leader on IBM's corporate strategy team, and a previous speaker at QCon. A key takeaway from the discussion was that there has been a steady increase in the adoption of asynchronous programming techniques and event-driven architectures. This adoption is the result of both a lower barrier to entry for implementing asynchronous patterns and a built-in benefit of increased system resilience. However, the flip side to event-driven architectures, and asynchronous systems, in general, is they are still difficult to reason about and understand. Which ties into the rise of designing for observability. In addition to sharing the report and the accompanying graph, Thomas Betts has also published an accompanying discussion with the team on an episode of the InfoQ podcast. Clare Liguori on Automating Safe and "Hands-Off" Deployments at AWS In this podcast, Clare Liguori, Principal Software Engineer at Amazon Web Services, joined InfoQ podcast co-host Daniel Bryant and discussed: the implementation of continuous delivery of public cloud software at AWS; the use of automation and deploying to multiple test environments; and the benefits of canary releasing. Every change that goes out to production at Amazon is code reviewed, and pipelines enforce this. With automated “full CD” there are no human interactions after a change has been reviewed and pushed into the source code repository before it gets deployed to production. Change Data Capture for Distributed Databases at Netflix In this recording of a QCon Plus talk, Raghuram Onti Srinivasan covers the challenges associated with capturing Change Data Capture (CDC) events from Cassandra and discusses the Flink ecosystem and the use of RocksDB. Netflix has microservices that use different kinds of databases based on the capabilities provided by each database. For example, when a new movie is added, the information is written to a Cassandra database. However, it may also be required by the user interface services, accessed with Elasticsearch, or be used within analytics engines. This talk focuses on the challenges involved in keeping the data synced from Cassandra, which is a distributed NoSQL database, to other databases using CDC. Why the Most Resilient Companies Want More Incidents Companies want more incidents "because companies want more learning". According to John Egan, co-founder and CEO at Kintaba, the incident management process is meant to be a cycle of the response and the account of the root cause, and the updating of internal processes and practices across the industry Egan, former co-founder and product lead of Workplace by Facebook, spoke about how tech organizations were doing incident management at QCon Plus May 2021. He recommended lowering the barrier to reporting incidents, holding effective incident review meetings using blameless postmortems, and giving everyone access to postmortem data. What Have We Learned Over the Last Decade of Microservices? This episode of the InfoQ podcast is a panel discussion from the microservices track QCon Plus held in May 2021. Track host Nicki Watt asks, "What have we learned over the last decade of microservices?" (A video recording of the discussion is also available.) The panelists included Chris Richardson, James Lewis, and Katie Gamanji. There were several great insights about how successfully developing, deploying, and maintaining resilient software depends as much or more on cultural and environmental factors than simply adopting microservices and all the tools and technology that now exist. Building Reliable Systems and Teaching SRE Apprentices In a recently published QCon Plus talk recording, Ana Margarita Medina, senior chaos engineer at Gremlin, shared how she has been using chaos engineering to build reliable systems. She also explored how chaos engineering can be used to decouple a system's weak points, learn from incidents, and improve monitoring and observability. In a related InfoQ podcast, Thomas Betts spoke with Tammy Bryant Butow, principal SRE at Gremlin, about training new site reliability engineers. The discussion covered the establishment of a formal SRE Apprenticeship program Bryant Butow led at DropBox and explored ideas about the best way to teach people new technical skills. There are benefits for the trainees, the mentors, and the company when people put in the effort to create a formal training program. Ballerina Swan Lake: 10 Compelling Language Characteristics for Cloud Native Programming Ballerina is an open-source programming language purpose-built for cloud native programming and integration. The language has been designed to make it easier to use, combine, and create network services and thereby allows you to integrate distributed applications seamlessly. In this recent InfoQ article, Dakshitha Ratnayake explores how the Ballerina language has come a long way with significant improvements since the 1.0 release in 2019. The latest Swan Lake release further simplifies building and deploying cloud-native microservices and applications through a network-aware and flexible type system, constructs for developing services and APIs (including REST, GraphQL, and gRPC), a sequence diagrammatic syntax, JSON support, and built-in concurrency among many other capabilities which will be explored in this article. Related to this, James Clark, the lead designer of the Ballerina programming language, joined Charles Humble on the InfoQ podcast episode: "James Clark on How Ballerina Handles Network Interaction, Data, and Concurrency". Service Mesh Ultimate Guide—Second Edition: Next Generation Microservices Development In this latest InfoQ Ultimate Guide, Srini Penchikala explores emerging architecture trends in the adoption of service mesh technologies, especially the multi-cloud, multi-cluster, and multi-tenant models. The guide discusses how to deploy service mesh solutions in heterogeneous infrastructures (bare metal, VMs, and Kubernetes) and explores application/service connectivity from the edge computing layer to the mesh layer. In a related podcast, William Morgan, CEO of Buoyant, sat down with Wes Reisz for the InfoQ podcast "Service Meshes and Linkerd with William Morgan". Case Study Turning Microservices Inside Out Bilgin Ibryam’s latest InfoQ article explores the concepts of turning microservices “inside out.” To future-proof microservices, they must be designed with inbound and outbound APIs, where the data flows through, and a meta API that describes these APIs. Ibryam argues that outbound events are turning into the preferred integration method for modern platforms such as cloud services, data sources, and even file systems. Custom-built microservices are not an exception, and emitting state change or domain events is the most natural way for modern microservices to fit uniformly among other event-driven systems. The responsibilities and the importance of meta APIs are growing as their scope changes from describing synchronous APIs to also including asynchronous APIs. The meta APIs are expanding toward enabling faster development cycles by ensuring safe schema evolution through compatibility checks, notifications for updates, code generation for bindings, test simulations, and so forth. Recent drivers such as cloud adoption and microservices architecture have accelerated the consolidation and community-driven standardization of open-source event streaming projects around Apache Kafka. This content is an excerpt from a recent InfoQ article written by Bilgin Ibryam: "Turning Microservices Inside-Out". Missed a newsletter? You can find all of the previous issues on InfoQ. Sponsored | | In this practical guide, distributed systems expert Lee Atchison shows you how to build apps that scale to handle unpredictable traffic with zero downtime. Learn the principles of service-based application architecture, how scaling affects the availability of your services Lambda platform, common design patterns for managing data in distributed applications, and more.
Learn more about this topic in the eBook "Architecting for Scale (By O’Reilly)" sponsored by Cockroach Labs. |
|
Upcoming events For practitioners by practitioners
QCon London 2022 (In-person): Don’t miss out on real-world insights to help you adopt the right technologies and practices.
QCon London (April 4-6, 2022) brings together the world’s most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices. Don’t miss your chance to connect in person with a global community of like-minded engineers. Attend in-person and save £205 if you register before January 10th.
QCon Plus 2022 (Online): Learn how to solve complex software engineering and leadership challenges from senior software developers.
At QCon Plus (May 10-20, 2022) online Software Conference you can expect technical talks from software leaders driving innovation and change, a focus on patterns and practices (not products and pitches), and implementable ideas you can use on after the event. Hear valuable insights and advice to help you with your learning journey. Save £300 if you register before January 10th.
InfoQ Live February 22: How do traditional security approaches scale in Cloud Native architectures? What does modern security look like both for platforms and infrastructure?
At InfoQ Live (February 22, 2022) you will learn practical advice from world-class DevSecOps & Modernizing Application Security Professionals on how you can overcome security challenges in the Cloud and especially in Serverless architectures. Discover practical ways to guide your problem-solving approach. Join us on Tuesday, February 22, 2022 (Tickets only $19.95). Senior software developers rely on the InfoQ community to keep ahead of the adoption curve. One of the main reasons software architects and engineers tell us they keep coming back to InfoQ is because they trust the information provided and selected by their peers. We’ve been helping software development teams adopt new technologies and practices for over 15 years through InfoQ articles, news items, podcasts, tech talks, trends reports, and QCon software development conferences. We hope you find this newsletter useful. If not, you can unsubscribe using the link below. Forwarded email? Subscribe and get your own copy. | 
