this post was submitted on 17 Aug 2023
12 points (100.0% liked)

DevOps

1675 readers
1 users here now

DevOps integrates and automates the work of software development (Dev) and IT operations (Ops) as a means for improving and shortening the systems development life cycle.

Rules:

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 1 year ago
MODERATORS
Ategon@programming.dev
RandomDevOpsDude@programming.dev
12
Is there software that tracks internal dependencies for CI/CD? (programming.dev)
submitted 1 year ago by jim@programming.dev to c/devops@programming.dev
5 comments fedilink hide all child comments
 

Here's a hypothetical scenario at a company: We have 2 repos that builds and deploys code as tools and libraries for other apps at the company. Let's call this lib1 and lib2.

There's a third repo, let's call it app, that is application code that depends on lib1 and lib2.

The hard part right now is keeping track of which version of lib1 and lib2 are packaged for app at any point in time.

I'd like to know at a glance, say 1 month ago, what versions of app is deployed and what version of lib1 and lib2 they were using. Ideally, I'm looking for a software solution that would be agnostic to any CI/CD build system, and doubly ideally, an open source one. Maybe a simple web service you call with some metadata, and it displays it in a nice UI.

Right now, we accomplish this by looking at logs, git commit history, and stick things together. I know I can build a custom solution pretty easily, but I'm looking for something more out-of-the-box.

top 5 comments
sorted by: hot top controversial new old
[–] zero@lemm.ee 3 points 1 year ago* (last edited 1 year ago)

It's probably more than you'd need for just two dependencies, but Apache's Ivy does more or less exactly that (although admittedly without a nice web UI)

https://ant.apache.org/ivy/

Another solution for simple builds could be to use git submodules to include your libraries, that way for any commit of the top level you can see exactly what commit of the lower levels you're using

[–] perviouslyiner@lemm.ee 2 points 1 year ago

This is one of the situations where subversion is better than git - svn "extern" can tag a specific revision of the library to be associated with each branch & revision of each project. Svninfo in each directory will tell your tools the version currently in each library the program is using.

[–] Lodra@programming.dev 2 points 1 year ago

I realize that you're looking for audit style information. Disclaimer: I'm not offering that 😅

You may be interested in automation that helps manage these dependencies and their version. Github offers dependabot as a service for repos on github.com. There's also renovate which is a little newer and I think has a better open source reputation. You can use them to fully automate dependency patching. Though I suspect most teams use these tools to automatically build PRs for a human to review and test.

[–] bahmanm@lemmy.ml 1 points 1 year ago* (last edited 1 year ago)

I believe you're already on the right track.

Your pipeline keeps track of the git commit that resulted in each build/deploy. You can use that (curl your CI/CD API and feed it into jq) to check out the build definition file for app (eg app/build.gradle) from that particular revision, and simply grep for lib1 and lib2. It should technically be possible to do this in a few lines of shell script.

[–] purelynonfunctional@programming.dev 1 points 1 year ago

The metadata you want is called a Software Bill of Materials, and there are a range of tools for generating them. Some generic ones include Trivy and Grype, but you may also find some for your language ecosystem by Googling ' + SBOM'.

One tool you can use to view these versions with a web UI is OWASP Dependency-Track.

All of the tools mentioned and linked above are F/OSS.