Developing & Testing Data pipelines locally with LocalStack!

Creating data pipelines and applications for the cloud comes with challenges like a complicated developer experience, dealing with managed cloud dependencies, and enduring long build times. These issues often disrupt your development and testing cycles. LocalStack's cloud emulation allows you to construct, deploy, and test data pipelines on your local machine. It facilitates integration testing of cloud solutions both locally and in CI pipelines. This approach saves time and money, enhances developer velocity, and supports high-quality, agile, test-driven development. In this talk, Harsh delves into developing and testing cloud-based data pipelines on your local machine. The session will provide a firsthand look at the new Snowflake emulator and demonstrate how you can use LocalStack to create Snowflake warehouses, databases, schemas, and tables, and integrate frameworks like Snowpark.

Related Talks

Supercharging CI with Dagger

Want to modernize your CI/CD workflows? 🚀 In this session, Jason McCallister introduces Dagger, the open-source programmable CI/CD engine that’s redefining how we build, test, and ship software. You'll learn what makes Dagger different from traditional CI/CD tools, how to write pipelines as code and run them locally, how to compose reusable, testable pipeline components, real-world examples of solving CI headaches with Dagger, and integration tips with Docker, Kubernetes, and beyond. Whether you’re a DevOps pro, platform engineer, or just tired of brittle YAML, this talk will show you how Dagger helps you ship faster and smarter.

Learn More
Learn More
Serverless Best Practices with AWS Lambda PowerTools

For one-off tasks, AWS Lambda really can be incredibly easy. You write a few lines of code, deploy it, and you have a function running in the cloud ready to respond to events, scale automatically, and that only costs you pennies. But as your application grows, so does some necessary complexity. When a few one-off functions become a full serverless backend architecture made up of interconnected services, you’ll need to pay careful attention to best practices to ensure that your application is easy to debug, maintain, and scale. That’s where AWS Powertools for Lambda fits in. It’s a suite of reusable utilities designed to simplify bringing best practices around things like logging, tracing, metrics, idempotency and more to your Lambda functions with minimal effort. This demo session will dive into some of the functionality provided by the AWS Powertools (TypeScript) core libraries, such as encapsulating best practices into reusable libraries for structured logging, metrics collection, idempotency, and more; leveraging Middy middleware to integrate common cross-cutting concerns, such as injecting Lambda context or automatically flushing metric; enabling local testing with LocalStack, allowing you to deploy and debug Lambda functions with structured logs, trace data, and embedded metrics; and providing modular examples that can be deployed to AWS or LocalStack with ease, enabling developers to explore libraries.

Learn More
Learn More
Step-up Authentication using Amazon Cognito on LocalStack

LocalStack Applications in Developer Hub provides sample templates to help LocalStack users adopt real-world scenarios to rapidly and conveniently create, configure, and deploy applications locally.

Getting started with Step-up-authentication demo

In this demo, we will setup a step-up authentication workflow for a higher level of security, deployed using Cloud Development Kit on LocalStack

Learn More
Learn More

Launch yourself in the world of local cloud development

Try for free
Try for free
Talk to Sales
Talk to Sales