Temporal

Overview

Temporal is a platform designed for developing and operating fault-tolerant applications, particularly beneficial for orchestrating complex business logic and long-running processes in distributed systems. It allows developers to define workflows as code using standard programming languages, abstracting the underlying complexities of state management, retries, and error handling. The core of Temporal's offering is the Temporal Platform, an open-source solution that consists of a durable execution engine (the Temporal Server) and client SDKs.

The platform maintains the complete state of a workflow, enabling it to recover from failures, including server crashes, network partitions, and application errors, without losing progress. This is achieved through Workflow Executions, which are durable, reliable, and independently versioned programs. Developers interact with the Temporal Server using SDKs available for languages such as Go, Java, TypeScript, Python, PHP, and Rust. These SDKs provide APIs for defining Workflow and Activity functions, which are the fundamental building blocks of Temporal applications.

Temporal's architecture is designed to address challenges common in microservice architectures, such as ensuring consistency across multiple services, handling asynchronous operations, and managing distributed transactions. By providing a durable execution environment, it allows developers to write sequential code that behaves as if it's running on a single, reliable machine, even when distributed across multiple processes and nodes. This approach aligns with principles of microservice decomposition, facilitating the management of inter-service communication and long-lived operations. For deployment, users can self-host the open-source Temporal Platform or utilize Temporal Cloud, a managed service offering the same capabilities.

Key features

• Durable Workflows as Code: Define complex, long-running processes using standard programming languages, with the platform handling state persistence and recovery automatically.
• Fault Tolerance and Reliability: Workflows automatically recover from failures (e.g., server restarts, network issues) without data loss or manual intervention, ensuring process completion.
• Automatic Retries and Timeouts: The platform manages retries for failed activities and applies timeouts, reducing the need for boilerplate error handling code in application logic.
• Developer SDKs: Supports multiple programming languages including Go, Java, TypeScript, Python, PHP, and Rust, allowing developers to write workflows in familiar environments.
• Scalability: Designed to handle millions of concurrent workflow executions, scaling horizontally to meet demand for high-throughput applications.
• Visibility and Debugging: Provides tools and APIs for monitoring workflow progress, inspecting state, and debugging failures in distributed systems.
• Version Management: Supports safe deployment of new workflow code versions without affecting in-flight executions, enabling continuous integration and delivery.
• Managed Cloud Offering: Temporal Cloud provides a fully managed service for the Temporal Platform, abstracting operational overhead.

Pricing

Temporal Cloud pricing is structured around resource consumption, specifically workflow and activity executions, storage for workflow history, and data transfer. A free tier is available for initial usage.

For detailed and up-to-date pricing information, refer to the Temporal Cloud pricing page.

Common integrations

• Databases: Temporal can integrate with various databases (e.g., PostgreSQL, MySQL) for storing application data that workflows interact with. Workflows themselves do not store application data directly but orchestrate services that do.
• Messaging Queues: Workflows can interact with message queues (e.g., Apache Kafka, RabbitMQ) to publish events or consume messages as part of their logic.
• Microservices: Designed for orchestrating calls between different microservices, providing reliability and state management across service boundaries.
• Cloud Services: Integrates with cloud-specific APIs (e.g., AWS S3, Google Cloud Storage, Azure Blob Storage) for data processing and storage tasks within workflows.
• Monitoring and Logging Tools: Supports integration with standard monitoring (e.g., Prometheus, Grafana) and logging (e.g., ELK Stack, Datadog) systems for observability into workflow executions (Temporal observability documentation).
• CI/CD Pipelines: Workflows can be part of CI/CD pipelines to automate deployment, testing, or other operational tasks.

Alternatives

• Cadence (Uber): An open-source fault-tolerant workflow engine, originally developed at Uber, which Temporal evolved from.
• Conductor (Netflix): A microservices orchestration engine from Netflix, known for its JSON-based workflow definitions and visual tooling.
• Zeebe (Camunda): A horizontally scalable workflow engine for microservices orchestration, part of the Camunda Platform, using BPMN for workflow modeling.
• AWS Step Functions: A serverless workflow service that allows orchestrating AWS services and other HTTP-based services using visual workflows.
• Azure Logic Apps: A cloud service for creating automated workflows that integrate apps, data, services, and systems across enterprises or organizations.

Getting started

The following example demonstrates a basic "Hello World" workflow in Go, defining a simple workflow that calls an activity to return a greeting.

package app

import (
	"context"
	"time"

	"go.temporal.io/sdk/client"
	"go.temporal.io/sdk/worker"
	"go.temporal.io/sdk/workflow"
)

// Workflow definition
func GreetWorkflow(ctx workflow.Context, name string) (string, error) {
	// Define activity options
	activityOptions := workflow.ActivityOptions{
		StartToCloseTimeout: time.Second * 5,
	}
	ctx = workflow.WithActivityOptions(ctx, activityOptions)

	var result string
	err := workflow.ExecuteActivity(ctx, GreetActivity, name).Get(ctx, &result)
	if err != nil {
		return "", err
	}

	return result, nil
}

// Activity definition
func GreetActivity(ctx context.Context, name string) (string, error) {
	return "Hello " + name + "!", nil
}

// Worker code to run the workflow and activity
func StartWorker(c client.Client, taskQueue string) {
	// Create a worker that can host both Workflow and Activity Functions.
	w := worker.New(c, taskQueue, worker.Options{})

	// Register the workflow and activity functions.
	w.RegisterWorkflow(GreetWorkflow)
	w.RegisterActivity(GreetActivity)

	// Start listening to the Task Queue.
	err := w.Run(worker.InterruptCh())
	if err != nil {
		// Log error
		return
	}
}

To run this example:

1. Set up a Temporal Server: This can be a local Docker setup or a connection to Temporal Cloud (Temporal Docker Compose setup).
2. Initialize a Temporal client: Create a client instance to connect to your Temporal Server.
3. Start a Worker: Run the StartWorker function, which registers the GreetWorkflow and GreetActivity with a specified task queue. The worker continuously polls the task queue for new tasks.
4. Start a Workflow Execution: From a separate client application, initiate an execution of the GreetWorkflow, passing the desired name. The Temporal Server will schedule the workflow, and the worker will pick it up, execute the activity, and return the result.

This setup allows developers to focus on the business logic within their workflows and activities, while Temporal handles the distributed systems challenges of state management, retries, and fault tolerance.