Product Launch
anonymous
62 points
48 comments
Posted29 days agoActive21 days ago
Dagger
dagger.iolauncheddeveloper_tooldevelopers
Key Features
define software delivery workflowsdefine dev environments
DevOpssoftware deliverydevelopment_tools
Key Features
But, the marketing heavily focuses on LLM stuff to the point of making everyone confused.
I currently manage my development environments via NixOS and Devenv, so if I could just keep that and achieve the same functionality, that sounds good to me.
Without the LLM bits, this is basically like Bazel or buck2, right?
I ended up opting for CUE and GitHub Actions, and I'm glad I did as it made everything much, much simpler.
Here are a few links to whet your appetite:
- https://cue.dev/docs/getting-started-with-github-actions-cue...
- https://cue.dev/docs/drying-up-github-actions-workflows/
- https://cue.dev/docs/spotting-errors-earlier-github-actions-...
Definitely read through the CUE documentation (https://cuelang.org/docs/), their YouTube videos (https://www.youtube.com/@cuelang/videos), and join the community Slack channel (https://cuelang.org/community/). I've gotten a lot of help in the Slack from both enthusiastic community members and from the developers themselves whenever I've gotten stuck.
To some extent yes. If all you have is 2 GitHub Actions YAML files you are not going to reap massive benefits.
I'm a big fan of CUE myself. The benefits compound as you need to output more and more artifacts (= YAML config). Think of several k8s manifests, several GitHub Actions files, e.g. for building across several combinations of OSes, settings, etc.
CUE strikes a really nice balance between being primarily data description and not a Turing-complete language (e.g. cdk8s can get arbitrarily complex and abstract), reducing boilerplate (having you spell out the common bits once only, and each non-commit bit once only) and being type-safe (validation at build/export time, with native import of Go types, JSON schema and more).
They recently added an LSP which helps close the gap to other ecosystems. For example, cdk8s being TS means it naturally has fantastic IDE support, which CUE has been lacking in. CUE error messages can also be very verbose and unhelpful.
At work, we generate a couple thousand lines of k8s YAML from ~0.1x of that in CUE. The CUE is commented liberally, and validation imported from native k8s types, and sprinkled in where needed otherwise (e.g. we know for our application the FOO setting needs to be between 5 and 10). The generated YAML is clean, without any indentation and quoting worries. We also generate YAML-in-YAML, i.e. our application takes YAML config, which itself is in an outer k8s YAML ConfigMap. YAML-in-YAML is normally an enormous pain and easy to get wrong. In CUE it's just `yaml.Marshal`.
You get a lot of benefit for a comparatively simple mental model: all your CUE files form just one large document, and for export to YAML it's merged. Any conflicting values and any missing values fail the export. That's it. The mental model of e.g. cdk8s is massively more complex and has unbounded potential for abstraction footguns (being TypeScript). Not to mention CUE is Go and shipped as a single binary; the CUE v0.15.0 you use today will still compile and work 10 years from now.
You can start very simple, with CUE looking not unlike JSON, and add CUE-specific bits from there. You can always rip out the CUE and just keep the generated YAML, or replace CUE with e.g. cdk8s. It's not a one-way door.
The cherry on top are CUE scripts/tasks. In our case we use a CUE script to split the one-large-document (10s of thousands of lines) into separate files, according to some criteria. This is all defined in CUE as well, meaning I can write ~40 lines of CUE (this has a bit of a learning curve) instead of ~200 lines of cursed, buggy bash.
If you had to pick, what's the "killer" feature for you with this module?
re: the cloud specifically see these GitHub issues:
https://github.com/dagger/dagger/issues/6486
https://github.com/dagger/dagger/issues/8004
Basically if you want consistently fast cached builds it's a PITA and/or not possible without the cloud product, depending on how you set things up. We do run it self-hosted though, YMMV.
If you have questions about Dagger, I encourage you to join our Discord server, we will be happy to answer them!
We shipped multi-language support because we had no choice. It was a major engineering effort that we hadn't originally planned for, but it was painfully obvious that remaining a CUE-only platform was suicide.
For a while there was activity on the #cue channel about a community SDK (that's how we got PHP, Java, Rust, Elixir and dotnet), but it didn't materialize.
It looks like you were in the minority that would have preferred to continue using the original CUE SDK - I'm sorry that we didn't find a way to continue supporting it.
What else could be used to abstract away your CICD from the launcher (Jenkins, Argo Workflows, GitHub Actions, etc.)?
They don't seem to have jumped for AI hype (yet?)...
I.e. declaratively setup a web of CI / deployment tasks, based on docker, with a code-first DSL, instead of the morass of copy-pasted (and yes orbs) CircleCI yaml files we have strewn about our internals repos.
But their DSL for defining your pipelines is ... golang? Like who would pick golang as "a friendly language for setting up configs".
The underlying tech is technically language-agnostic, just as aws-cdk's is (you can share cdk constructs across TypeScript/Python), but it's rooted in golang as the originating/first-class language, so imo will never hit aws-cdk levels of ergonomics.
That technical nit aside, I love the idea; ran a few examples of it a year or so ago and was really impressed with the speed; just couldn't wrap my around "how can I make this look like cdk".
https://docs.dagger.io/cookbook/services?sdk=typescript
Still looks like "a circa-2000s Java builder API" and doesn't look like pleasant / declarative / idiomatic TypeScript, which is what aws-cdk pulled off.
Genuinely impressively (imo), aws-cdk intermixes "it's declarative" (you're setting up your desired state) but also "it's code" (you can use all the usual abstractions) in a way that is pretty great & unique.
https://gist.github.com/stephenh/8c7823229dfffc0347c2e94a3c9...
Like I'm still building a DAG, but by creating objects with "kinda POJOs" (doesn't have to be literally POJOs) and then stitching them together, like the outputs of 1 construct (the build) can be used as inputs to the other constructs (tests & container).
/s I've never heard of Dagger the DI framework but I have heard of this Dagger. Names will overlap sometimes and it's not a big deal.
Android is the most popular operating system on earth. Names can overlap but you shouldn’t choose one where you can’t live up to owning it.
> can’t live up to owning it.
?
> > can’t live up to owning it.
> ?
Dagger, the DI framework, is very popular, supported by Google, and the primary way to do DI on Android (for many). It is not going anywhere. It is unlikely that this tool or product will create a bigger audience. Therefore, it might never "live up to the name," compared to the benchmark set by current Dagger. It's a simple concept.
But one flaw (IMO) that it can’t export artifacts and import into other steps without breaking the cache.
Eg if you provide monorepo as input, and then on some step narrow your build to one specific dir, then even when files change outside of that dir then caching still is invalidated.
Which makes it extremely verbose and maintenance nightmare to keep multiple narrow inputs and keep all those paths up to date.
It serves a place where a dockerfile is not enough, and CI workflows are too difficult to debug or reason about.
I do have some current problems with it though:
1. I don't care at all about the LLM agent workflows, I get that it is possible, but the same people that chose dagger for what it was, is not the same audience that runs agents like that. I can't choose dagger currently, because I don't know if they align with my interests as an engineer solving a specific problems for where I work (delivering software, not running agents).
2. I advocated for modules before it was a thing, but I never implemented it. It is too much magic, I want to write code, not a DSL that looks like code, dagger is already special in that regard, to modules takes it a step too far. You can't find the code in their docs anymore, but dagger can be written with just a .go, .py or .rs file. Simply take in dagger as a dependency and build your workflow.
3. Too complex to operate, dagger doesn't have runners currently, and it is difficult to run a setup in production for CI yourself, without running it in the actions themselves, which can be disastrous for build times, as dagger often leads you into using quite a few images, so having a cache is a must.
Dagger needs to choose and execute; not having runners, even when we we're willing to throw money at them was a mistake IMO. Love the tool, the team, the vision but it is too distracted, magical and impatient to pick up at the moment.
1. Yes we got over-excited with the agent runtime use case. We stand by the LLM implementation because we never compromised on the integrity of Dagger's modular design. But our marketing and product priorities were all over the place. We're going to refocus on the original use case: helping you ship software, and more particularly building & testing it.
2. Modules have warts but they are essential. We will continue to improve them, and remain committed to them. Without this feature, you have to write a complete standalone program every time you want to build or test your software. It's too much overhead.
3. Yes you are right. We really thought we could co-exist with CI runners, and get good performance without reinventing the wheel. But for various reasons that turned out to not be the case. So we're going to ship vertically integrated runners, with excellent scalability and performance. DM me if you want early access :)
TLDR: yes we needed to choose and execute. We have, and we are.
Thank you again for the feedback.
Best of luck and thx for taking my harsh feedback in strides!
Then... it wasn't. The more I read the less I ever want to see this again. The LLM train has got to end at some point.
I think a more interesting point of comparison is the Claude Code Github Action, Co-Pilot code reviews, etc.
Although I'm not sure if that's so much a value-added? It's not so hard to just create a container and launch an agent in it.
The whole interesting thing was to use actual programming languages for Docker build, which I think was what they initially tried to do, but now it's a bit incomprehensible... I guess conceptually Dagger relates to Dockerfile a bit like Pulumi related to Terraform?
Not affiliated with Hacker News or Y Combinator. We simply enrich the public API with analytics.