RSS

2021-05-28: IAC & DevX Team Status Update

By carabasdaniel |
Categories:

We had a bit of a slow week today. We have a few company-wide mental health holidays and one was this week. It’s really good to disconnect from it all sometimes.

Community Contributions

We’d like to thank the following people in the Puppet Community for their contributions over this past week:

Check all the ways to reach us if you want to directly contact us about anything module related.

New Module / Gem Releases

The following modules were released this week:

Puppet Content Templates LA Release Immenient

You may have seen the PDK user experience survey that went around some weeks ago. We wanted to gain your feedback to validate some assumptions we had about creating content for Puppet products. One of the hypothesis that we started with was a prediction that the monolithic pdk-templates repository was too complex and inflexible for end users.

The results of the survey bore this out, and so we started work on a new approach to content templating.

Instead of a single repository which you’d need to fork and maintain to keep up to date, we built out a modular system that would allow you pick and choose the components you’d like to include.

We’re happy to report that we are making great progress and will have a limited availability release out very soon for you to experiment with, so watch this space for updates!

DSC And the PDK Go Prototype

The new PDK approach that the DevX team is working on is implemented in Go and screamingly fast. Since their LA release is imminent, this week felt like a good time to see what it will be like to use the new templating in a real project.

We put up a prototype PR for switching to the new templating (when it becomes available) in our Puppet.Dsc module.

How well did it work?

Well, it decreased the time to scaffold a minimal Puppet module from thirty seconds to less than one second.

It’s hard to overstate what a massive improvement this is! In our next milestone for Puppet.Dsc (after the current milestone, which will see us release 1.0.0 to the PowerShell Gallery!), we’ll be setting up automation to do a few things:

  1. Automatically publish any new release of a PowerShell module with DSC Resources on the gallery - either a new version or a new module - within 24 hours of release, straight to the Forge.
  2. Automatically rebuild and publish the incremented build of all previously puppetized PowerShell modules with DSC Resources to the Forge. It’s that second use case, the rebuilding, that makes this change in templating extremely effective: There are over 350 PowerShell modules with DSC Resources on the Gallery right now. Even if we only rebuilt the latest version of those modules, we would be saving almost three hours of build time.

But when we make updates to the Puppet.Dsc (as it is the core builder, which means we made changes to how PowerShell modules with DSC Resources are puppetized), we’re actually rebuilding older module versions! That means even if we are extremely conservative with our estimations and pretend the PowerShell modules only average 5 releases each, we’re going to shave a frankly astounding 875 hours from our build time!

The real time to rebuild and publish is, of course, shorter because we will be running this all in parallel; but saving more than a literal month’s worth of build time on every release of Puppet.Dsc is an overwhelming win for us (and for the folks who maintain the Github Actions infrastructure)!

Kubernetes certification updates

We’re one step closer to finishing the certification for our puppetlabs/kubernetes module for Kubernetes 1.20. We’ve now got an automated complete k8s stack using CloudFormation and coming down to the finish line with Sonobuoy conformance testing!

Cloud CI Update

This week we have finally closed the training and education milestone. We will continue now with the Puppet Developer Experience improvement milestone for Cloud CI and we have already started the P.o.C (Proof of Concept) for developer box provisioning. This allows a module developer to request an instance representative of the team’s testing environments using just a GitHub token.

This is kind of a big deal, because many developer environments used for validatiting Puppet modules can be pretty complex–think of a Splunk architecture with forwarders, indexers, search heads, etc. We will continue the work in close cooperation with our security team to make this a secure and useful solution for Puppet module developers.