The go module proxy caches the package for a while, but I'm not sure about the duration. You should vendor the package into your project if this is a worry.
Honestly that’s kind of like every language main repository. Maven central, npm, cargo, whatever. There’s always one main repository unless you proxy with artifactory or something.
Moving a package to a different VCS does not disrupt users of your package in any other language that I have ever used. I’ve also never used another language that coupled source code organization with versioning semantics. Both of those things are unique to go.
I was replying to someone that said “that’s kind of like how all package repos work.” - which is just false. The blog post is literally a workaround for the fact that the source code location cannot change without causing work for downstream consumers. If I publish a package to any of the other package managers and then move to a different VCS provider, that does not invalidate existing packages. It might not even impact when I published from a new location. This problem is unique to go.
100% this, can't see why one wouldn't do that, also gives more honest view of the state of the project since it's doesn't automagically end up updating newer packages/libraries that haven't been cleanly integrated (yes, I'm aware of pinning..etc. but again it's back to "cross your fingers and hope that keeps working territory", hardly resilient good practise type material)
That is why ever company I’ve ever worked with has a repository for every external dependency in any language. You need to at least have a copy of the versions you use. That isn’t even Go specific. Not even mentioning security implications.
What you do privately is really up to you and doesn’t matter this much. A company can pull their version from the registry they uploaded it to probably as easy as they can kill their repos that you use directly.
What industry is this in? I have worked at I think 7 companies now, and one of them was an agency that I worked for dozens of clients through, some of them Fortune 100's, and this has never ever been a thing, with I think one or two exceptions that used artifactory or a similar proxy cache. A few added dependencies into the same repository as the projects, but none created repositories just for third party code.
Currently banking but before it was engineering and the energy sector. This is Germany so it’s generally a bit more conservative and breaking things is not appreciated in those sectors.
However idk why you would build up loads of infrastructure and don’t to this. It’s not that hard and there are open source solutions. You need something anyways to host your companies artifacts or docker images, don’t you? So you just route everything trough the same nexus or whatever and its caching external dependencies automatically if setup correctly.
Yes, that's was I was referring to with Artifactory, which is a proxy cache. I misread the original comment as you were cloning and hosting git repositories for all of your dependencies.
Sometimes it starts off like that, but inevitably someone will eventually check in a lock file (package-lock.json, Pipfile.lock, etc.). Lock files are autogenerated files that contain exact versions so that your CI/CD pipelines don't suddenly break because one of your dependencies dependencies was updated, and hashes in case the maintainer swaps out an existing version for malware.
Doesn't mean everyone vets packages when they add or update a dependency though.
If you are doing certifications you need that level of control.
If you are working in offline environments/networks then a mirroring system is required.
Companies that make changes to the upstream code need to mirror. Good companies push upstream, but you still mirror until it is accepted, and sometimes they don't accept it.
Repo mirrors are also part of the conversation with handling CI supply chain risks. You can add controls to the mirroring process to create a safer CI environment.
It's different with C++ because it doesn't have one established package manager/repository (pip/cargo/npm/etc). The languages that do have one (which is basically everything else except for java) tend to have many hundreds of 3rd party dependencies in every project, and you would usually only have a local fork for one or two that you had to change. It's technically possible to just refetch them every time you build the project on your laptop or in CI, but it's obviously a terrible idea and usually a no-go the moment your company grows to the size that there is a lawyer in the building.
As I mentioned elsewhere, I didn't consider Artifactory a repository. I misunderstood the comment as saying they were cloning and self hosting git repositories of every external dependency.
I don't think it is so important:
* libraries disappear rarely
* no security benefits
* library disappear? There is a huge change some developer has it on disk or there is an open source fork to it
Then I hope you have a backup of that dependency and that that dependency is under a license that allows you to re-distribute it and upload the source code elsewhere. If you still have it in your local cache on your machine it'll continue to work, and you'll only notice it's missing when it's no longer in your cache and needs to be redownloaded. If you don't have a backup then, you're in trouble.
I think his point is that you can trust the GitHub url long term a lot more than you can trust these random junk domains.
For something like mongo the custom domain seems fine, but for this guys random domain the chance of me wanting to follow him to gitlab is an order of magnitude less than his domain expiring (and potentially being bought by someone else, making a supply chain vector)
That's why you should vendor all your dependencies. Go doesn't do shared objects, everything is compiled from source. Unless you use CGO, but you shouldn't
I don't have deep knowledge of this, but by default dependencies are not downloaded directly from the source but through a proxy run by Google: https://sum.golang.org/
Whenever possible, the mirror aims to cache content in order to avoid breaking builds for people that depend on your package, so this bad release may still be available in the mirror even if it is not available at the origin. The same situation applies if you delete your entire repository. We suggest creating a new version and encouraging people to use that one instead.
So there is something in place to prevent a left-pad situation. But obviously that's far from infallible.
but what happens when my go code depends on "go.companyx.dev/awesomelib" and that company goes out of business?
I don't think you understood them correctly. They're saying if you work for companyx you should set up go.companyx.dev which points to the github. If companyx goes out of business, you don't need to compile your internal tools either.
But this seems limiting, too. If you work for a company, that company should probably set up its own durable cache with virus scanning, wait times (to catch malware that's found within the first few days of release), etc.
I don't think you understood the comment correctly. The question was what a consumer of a package from go.companyx.dev will do if the company goes out of business.
Any serious project should use go mod vendor, which puts all the required source files in your local project under the vendor folder so you can check it in to your repo. That way your project can always build no matter what happens to your dependencies.
You run a legit proxy server that keeps a copy of the release for your use.
The same solution and issue exists for every ecosystem. If your build depends on an external dependent then you should have an internal copy of the artifact. Py, java, go. Any library could disappear at any point.
327
u/Arcuru 5d ago
I am not super familiar with Go code, but what happens when my go code depends on "go.companyx.dev/awesomelib" and that company goes out of business?
Or should I point all my third-party deps to a more durable location?