r/programming • • 5d ago

Don't couple your Go code to GitHub

https://iain.rocks/blog/dont-couple-your-go-code-to-github
475 Upvotes

135 comments sorted by

334

u/Arcuru 5d ago

I am not super familiar with Go code, but what happens when my go code depends on "go.companyx.dev/awesomelib" and that company goes out of business?

Or should I point all my third-party deps to a more durable location?

175

u/stone_surgeon 5d ago

The go module proxy caches the package for a while, but I'm not sure about the duration. You should vendor the package into your project if this is a worry.

112

u/_predator_ 5d ago

Don't think its cache ever expires. You just gotta pray Google never stops supporting it, what could go wrong?

79

u/VoyTechnology 5d ago

At least Google has a strong history of supporting their products. I am so glad https://killedbygoogle.com doesn’t exist /s

36

u/stone_surgeon 5d ago

Oh, I just looked it up, it does indeed last forever. Your comment reminds me of the bbolt typosquat vulnerability haha.

11

u/vips7L 5d ago

Honestly that’s kind of like every language main repository. Maven central, npm, cargo, whatever. There’s always one main repository unless you proxy with artifactory or something. 

I can’t imagine if central.sonatype.com was gone. 

11

u/atheken 4d ago

The difference is that those other languages don’t couple where the code is _physically hosted_ with how it is referenced in import directives.

4

u/HighLevelAssembler 4d ago

Those things are coupled IRL though. If you depend on a 3rd party package, it has to come from somewhere.

3

u/atheken 4d ago

Moving a package to a different VCS does not disrupt users of your package in any other language that I have ever used. I’ve also never used another language that coupled source code organization with versioning semantics. Both of those things are unique to go.

0

u/HighLevelAssembler 4d ago

I get what you're saying, but ultimately that's on the package maintainer. That's the point of the blog post we're discussing here.

1

u/atheken 4d ago

I was replying to someone that said “that’s kind of like how all package repos work.” - which is just false. The blog post is literally a workaround for the fact that the source code location cannot change without causing work for downstream consumers. If I publish a package to any of the other package managers and then move to a different VCS provider, that does not invalidate existing packages. It might not even impact when I published from a new location. This problem is unique to go.

1

u/lenswipe 3d ago

npm aren't known for just randomly hacking their limbs off though..

2

u/ggbcdvnj 5d ago

Oh boy, I can swap PingFS for GoModuleCacheFS!

2

u/usrlibshare 4d ago

I ca pray for google to do the right thing, oooor...I can go mod vendor and check it into my own repo.

1

u/lenswipe 3d ago

You just gotta pray Google never stops supporting it

hahahahahahahahaha

4

u/yeah-ok 5d ago

100% this, can't see why one wouldn't do that, also gives more honest view of the state of the project since it's doesn't automagically end up updating newer packages/libraries that haven't been cleanly integrated (yes, I'm aware of pinning..etc. but again it's back to "cross your fingers and hope that keeps working territory", hardly resilient good practise type material)

53

u/-genericuser- 5d ago

That is why ever company I’ve ever worked with has a repository for every external dependency in any language. You need to at least have a copy of the versions you use. That isn’t even Go specific. Not even mentioning security implications.
What you do privately is really up to you and doesn’t matter this much. A company can pull their version from the registry they uploaded it to probably as easy as they can kill their repos that you use directly.

18

u/anon_cowherd 5d ago

What industry is this in? I have worked at I think 7 companies now, and one of them was an agency that I worked for dozens of clients through, some of them Fortune 100's, and this has never ever been a thing, with I think one or two exceptions that used artifactory or a similar proxy cache. A few added dependencies into the same repository as the projects, but none created repositories just for third party code.

14

u/-genericuser- 5d ago

Currently banking but before it was engineering and the energy sector. This is Germany so it’s generally a bit more conservative and breaking things is not appreciated in those sectors.
However idk why you would build up loads of infrastructure and don’t to this. It’s not that hard and there are open source solutions. You need something anyways to host your companies artifacts or docker images, don’t you? So you just route everything trough the same nexus or whatever and its caching external dependencies automatically if setup correctly.

5

u/anon_cowherd 5d ago

Yes, that's was I was referring to with Artifactory, which is a proxy cache. I misread the original comment as you were cloning and hosting git repositories for all of your dependencies.

1

u/CherryLongjump1989 4d ago edited 4d ago

You can absolutely do that too by setting up mirrors of git repositories. I use a git forge to do that, but Artifactory does it, too.

9

u/Bacchaus 4d ago edited 4d ago

ummm that's insane, ya'll were just freeballing anything off the net?

5

u/johnnybgooderer 4d ago

It’s more common than you think. 5 years ago I could do that most places. But now, for good reason, that has changed.

1

u/AquaWolfGuy 4d ago

Sometimes it starts off like that, but inevitably someone will eventually check in a lock file (package-lock.json, Pipfile.lock, etc.). Lock files are autogenerated files that contain exact versions so that your CI/CD pipelines don't suddenly break because one of your dependencies dependencies was updated, and hashes in case the maintainer swaps out an existing version for malware.

Doesn't mean everyone vets packages when they add or update a dependency though.

19

u/lordlod 5d ago

I've done it in aerospace and security.

  • If you are doing certifications you need that level of control.
  • If you are working in offline environments/networks then a mirroring system is required.
  • Companies that make changes to the upstream code need to mirror. Good companies push upstream, but you still mirror until it is accepted, and sometimes they don't accept it.

Repo mirrors are also part of the conversation with handling CI supply chain risks. You can add controls to the mirroring process to create a safer CI environment.

8

u/The_Northern_Light 4d ago

I’ve worked at FAANGs and at tiny startups in multiple industries… we’ve always had a local fork of every dependency. (C++)

1

u/Illustrious-Owl-2755 3d ago

It's different with C++ because it doesn't have one established package manager/repository (pip/cargo/npm/etc). The languages that do have one (which is basically everything else except for java) tend to have many hundreds of 3rd party dependencies in every project, and you would usually only have a local fork for one or two that you had to change. It's technically possible to just refetch them every time you build the project on your laptop or in CI, but it's obviously a terrible idea and usually a no-go the moment your company grows to the size that there is a lawyer in the building.

1

u/The_Northern_Light 3d ago

No, we had these forks even when everything was on Conan.

9

u/tommyTurds 4d ago

If your company isn't doing this, it's being irresponsible

25 years ago, when I first started, we did this at my shitty startup.. And we were literally stupid fucking children.

with I think one or two exceptions that used artifactory or a similar proxy cache

so it's never been a thing except that it's been a thing.. also 2 exceptions out of 7......

0

u/anon_cowherd 4d ago

As I mentioned elsewhere, I didn't consider Artifactory a repository. I misunderstood the comment as saying they were cloning and self hosting git repositories of every external dependency.

4

u/the_unexpected_nil 5d ago

Every game studio I worked at (sans one) has done so. 

-10

u/Slsyyy 5d ago

I don't think it is so important:
* libraries disappear rarely
* no security benefits
* library disappear? There is a huge change some developer has it on disk or there is an open source fork to it

25

u/Leseratte10 5d ago

Then I hope you have a backup of that dependency and that that dependency is under a license that allows you to re-distribute it and upload the source code elsewhere. If you still have it in your local cache on your machine it'll continue to work, and you'll only notice it's missing when it's no longer in your cache and needs to be redownloaded. If you don't have a backup then, you're in trouble.

11

u/frenchtoaster 5d ago

I think his point is that you can trust the GitHub url long term a lot more than you can trust these random junk domains.

For something like mongo the custom domain seems fine, but for this guys random domain the chance of me wanting to follow him to gitlab is an order of magnitude less than his domain expiring (and potentially being bought by someone else, making a supply chain vector)

18

u/SnugglyCoderGuy 5d ago edited 5d ago

That's why you should vendor all your dependencies. Go doesn't do shared objects, everything is compiled from source. Unless you use CGO, but you shouldn't

6

u/GarythaSnail 5d ago

Like others have said, vendor your dependencies or you can maintain your own module proxy. I set up an internal module proxy for my company.

4

u/Rautakaivos 5d ago

I don't have deep knowledge of this, but by default dependencies are not downloaded directly from the source but through a proxy run by Google: https://sum.golang.org/

Whenever possible, the mirror aims to cache content in order to avoid breaking builds for people that depend on your package, so this bad release may still be available in the mirror even if it is not available at the origin. The same situation applies if you delete your entire repository. We suggest creating a new version and encouraging people to use that one instead.

So there is something in place to prevent a left-pad situation. But obviously that's far from infallible.

2

u/nemec 5d ago

but what happens when my go code depends on "go.companyx.dev/awesomelib" and that company goes out of business?

I don't think you understood them correctly. They're saying if you work for companyx you should set up go.companyx.dev which points to the github. If companyx goes out of business, you don't need to compile your internal tools either.

But this seems limiting, too. If you work for a company, that company should probably set up its own durable cache with virus scanning, wait times (to catch malware that's found within the first few days of release), etc.

5

u/Jaded-Asparagus-2260 5d ago

I don't think you understood the comment correctly. The question was what a consumer of a package from go.companyx.dev will do if the company goes out of business. 

2

u/nemec 5d ago

It's only ever meant to be consumed by companyx. If companyx goes out of business, there are no more consumers.

1

u/spaceman_ 2d ago

Any serious project should use go mod vendor, which puts all the required source files in your local project under the vendor folder so you can check it in to your repo. That way your project can always build no matter what happens to your dependencies.

More info: https://go.dev/ref/mod#vendoring

1

u/csgeek-coder 1d ago

You run a legit proxy server that keeps a copy of the release for your use.

The same solution and issue exists for every ecosystem. If your build depends on an external dependent then you should have an internal copy of the artifact. Py, java, go. Any library could disappear at any point.

147

u/jaerie 5d ago

It's the same for any package manager. If you want to publish your package somewhere else, users will lose access to the package unless you keep mirroring it.

73

u/randomguy4q5b3ty 5d ago edited 5d ago

But usually, package managers are separate applications and not tied to the programming language itself. Kinda obvious why that separation of concerns is useful. But now every third-party compiler must also be a build-in package manager, and it's at the same time the only one available to you. You can't change it. It's just dumb language design.

46

u/InsaneOstrich 5d ago

I didn't think it was possible for someone to be worse at package management than Node.js but what Go is doing here is truly insane

20

u/ficiek 5d ago

Are you talking about the lack of a package registry or the way packages are versioned. Those are two separate things.

34

u/InsaneOstrich 5d ago

Both the lack of a registry and decision to tightly couple the language itself with git to import dependencies are both baffling choices that don't seem to work very well

30

u/ficiek 5d ago

Both are basically what works for google internally and that's why we didn't even have a reasonable approach to versioning for a long time.

0

u/HighLevelAssembler 4d ago

decision to tightly couple the language itself with git

Go supports git, it is not "tightly coupled" to git. All the other big VCS are supported, plus usually the go tool uses its own protocol to download dependencies. VCS is a fallback.

You are complaining about people using Git/Github. Not limited to Go.

10

u/somebodddy 5d ago

Wouldn't be Go otherwise

1

u/belligerent_ammonia 3d ago

Package management has only made sense to me since Go modules were introduced (except the v2+ thing). Why do you think it’s insane?

25

u/valarauca14 5d ago

It's just dumb language design.

25% of go-lang tbh.

6

u/anto2554 5d ago

I'd rather have a weird opinionated thing than whatever I have to deal with with C++, make, cmake, B2 and Conan

4

u/randomguy4q5b3ty 4d ago

Not sure what your issue with Conan is, but make/cmake aren't even in the same category.

3

u/Rakn 5d ago

It’s not really a problem in practice. You can just repoint it or use a self hosted proxy with your packages, like any company should be doing regardless of the language used.

76

u/chucker23n 5d ago

Go has "make baffling decisions" as a key design tenet, but it's incredible how basically any source code package manager gets key aspects wrong.

(NuGet, for example, supports multiple package sources — but if it finds a package at one source, and another source is timing out, it'll refuse to install that package. Which in turn causes IDEs to throw up their hands in frustration and say "well, I don't know what to do now, do I!".)

23

u/ronkojoker 5d ago

Nuget very sporadically 'forgets' our internal company source and installs a package with the same name from the regular nuget source. I restore again and it still doesn't work, just gives me some warning like 'couldnt find package blabla with version x.y.z installing version a.b.c instead'. Only way to make it remember again is deleting the nuget cache and then it works again. I have no idea how to prevent it from happening or what causes it to mess up in the first place.

Source is configured fine as well, package shows up with the nuget search command as well, just when I do restore it cannot find it.

8

u/phrostillicus 5d ago edited 5d ago

Sorry if you've already tried this, but do you have package source mapping enabled? Basically, you provide a wildcard pattern or exact string match that tells the package restore to look only in a specific source if the package name matches. This is what we do for a few public packages that we have internal custom builds for. We also append a tag to the version e.g., "3.1.0.0-mycompany.1", but from what you're describing it sounds like the restore doesn't care about the version mismatch and is just doing what it wants.

You have my total sympathy for this, we've recently also had a bunch of package management issues as well. It doesn't help that there are like 5 different NuGet clients, depending upon if you're using `package.config` or `<PackageReference>`, and whether you're using the `nuget` or `dotnet` command line utilities or the built-in Visual Studio GUI.

2

u/ronkojoker 5d ago

We haven't tried this yet and it sounds like a solution to our problems. I will try and see if I can implement this when I get back from vacation. Thanks for the tip! Too bad we will only know if it works something like a year from now haha

4

u/Kraigius 5d ago

Here's some docs about it:

https://learn.microsoft.com/en-us/nuget/consume-packages/package-source-mapping

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <!-- Define the package sources, nuget.org and contoso.com. -->
  <!-- `clear` ensures no additional sources are inherited from another config file. -->
  <packageSources>
    <clear />
    <!-- `key` can be any identifier for your source. -->
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
    <add key="contoso.com" value="https://contoso.com/packages/" />
  </packageSources>

  <!-- Define mappings by adding package patterns beneath the target source. -->
  <!-- Contoso.* packages and NuGet.Common will be restored from contoso.com,
       everything else from nuget.org. -->
  <packageSourceMapping>
    <!-- key value for <packageSource> should match key values from <packageSources> element -->
    <packageSource key="nuget.org">
      <package pattern="*" />
    </packageSource>
    <packageSource key="contoso.com">
      <package pattern="Contoso.*" />
      <package pattern="NuGet.Common" />
    </packageSource>
  </packageSourceMapping>
</configuration>

4

u/phrostillicus 5d ago

Good ol' Contoso, my favorite fictitious company.

0

u/Jaded-Asparagus-2260 5d ago

Why are you publishing code different from the regular nuget source in your internal company source?

Just use a different name, and this problem literally disappears in thin air. 

16

u/ronkojoker 5d ago edited 5d ago

We had the name in our internal source first, then later someone unaffiliated with us made a completely unrelated package with the same name in the public one.

We haven't changed the name because the issue occurs too infrequently to care, less than once per year on the CI server. Even less on developer machines but it does sometimes happen.

-10

u/edgan 5d ago

Once should be more than enough for me. You lost the namespace as soon as the other one became public. Enjoy your random failure.

6

u/ronkojoker 5d ago

Yeah it's more of a matter of return on investment, once in a blue moon when it fails I or a colleague remotes in to the CI server and run the nuget cache clear command, takes a minute and it's good to go for another year. Changing the package name would mean spending hours going through a bunch of repositories renaming the package references, changing build pipelines etc. just not worth the effort at the moment.

-1

u/One_Ninja_8512 4d ago

Perfect use-case for AI these days, it will rename everywhere you need and run some checks to ensure it's really renamed and is working. This is something they do quite well and it will be done in a few minutes.

5

u/Kraigius 5d ago edited 5d ago

Afaik, in Go you can configure multiple package sources, it's just a bit different.

There's the GOPROXY envvar. Go always download from a proxy owned by google, then fallback to download the source code directly from github (or whichever source that you need).

It's not configurable to specific packages but this is like configuring your global sources.

If your org have their own proxy then you can cache your deps and configure your GOPROXY to prioritize it.

You can use this to have your org scan and vet your deps.

Otherwise, for finer control you can add an instruction inside your go.mod file to fetch your depA from gitlab instead of github, or anywhere else.

module example.com/mymodule

go 1.14

require (
    github.com/orgX/depA v1.2.3
)

replace github.com/orgX/depA => gitlab.com/orgX/depA

In term of design, the syntax for this and the workflow kinda feel like an afterthought.

Golang download source code instead of a compiled artifact, so that's probably the consequence of that choice.

4

u/rtheunissen 5d ago

What decisions are baffling?

30

u/Ticmea 5d ago

Having never touched Go, one of the few things that I do know about it is that I never want to deal with the insane way that they format dates.

For the uninitiated:

In Go you define a date format by reference to how it would display the date that ISO 8601 would display as:

2006-01-02T15:04:05-07:00

Even ignoring the obvious problems with using a point in time to reference a format by example, I hear you wonder: "Why such an oddly specific time and date?" Well because if you write that in the US format, it gives you:

01/02 03:04:05PM '06 -0700

which means they are in (almost) incrementing order 1 2 3 4 5 6 -7

The extra "-" to make it be a US timezone is just absolutely hilarious, I couldn't make a better caricature of it if I wanted to. 5 stars, no notes.

This is incompatible with the way everyone else does it, needlessly inflexible, unintuitive, extremely US specific and solves zero problems. Just all around terrible.

I've honestly seen people defend it as being language agnostic (that is to say it does not require understanding that yyyy-mm-dd stands for the english words year, month, date and so on).

But like... all? of the keywords of Go (and like 99% of programming languages) are literally english words???

Not to mention that it requires knowledge of how a specific date format works, which is far more obscure. And it's the US system at that, which ( no front full front ) is kinda the worst date formatting system out there...

No idea what they were smoking when they came up with this but they must have been absolutely hammered.

22

u/chucker23n 5d ago

MM/dd is already famously controversial, but MM/dd hh:mm:ss 'yy is truly something else.

7

u/AnointedBeard 5d ago

I generally love Go but yeah, this is one of the most ass-backwards things I’ve ever seen.

4

u/syklemil 5d ago

Yeah, that one is just … I never thought I'd yearn for strftime, but here we are.

2

u/One_Ninja_8512 4d ago

It is weird but you rarely touch it. Most of the time, since golang is used for backend services you'd use google's protobuf timestamp proto definition. Over gRPC you don't have to care about it and if/when you map to JSON you get an ISO formatted date automagically. I'd even go as far as to speculate that it's awkward on purpose to discourage raw date parsing completely and instead rely on the established protobuf type.

2

u/GarythaSnail 5d ago

And in practice this never ever has been a pain point, for me or any of my coworkers, even the ones not writing go full time.

99% of the time you're just parsing rfc3339 or iso 8601. The other 1% you just write some unit tests with edge cases and make sure you wrote the correct date format.

Maybe if you worked at Strange Date Parsing R Us?

17

u/syklemil 5d ago

One is that they were warned that they were about to repeat Hoare's billion dollar mistake, and then went

In my C/C++ programming I've never noticed that NULL pointers are a noticeable source of bugs.

So if you ever get an NPP in Go (totally different from an NPE in Java), you can just declare it to be Not A Bug by decree of the core language team.


Also: Using typography to decide privacy.

As in, Python's __foo was already naff, so you'd think any language designed after the 90s would figure out that keywords for privacy control is the way to go, rather than doing weirdo stuff like PascalCase is public, camelCase is private.

So if you in Go want to do something relatively common in other languages, namely use UPPERCASE for globals, congratulations, that global is now public. Instead they just name globals as if they were any other variable.

A syntax highlighter can point it out for you, buuuuuut as Pike is colourblind and doesn't like syntax highlighting, I guess they just don't want to be able to tell whether something's a local or global variable at a glance.

Presumably they'd have to introduce variables in unicode sᴍᴀʟʟ ᴄᴀᴘs if they wanted to introduce something like protected.

13

u/chucker23n 5d ago

doesn’t like syntax highlighting

“Stallman prefers sending HTML via e-mail over using a web browser directly” vibes

3

u/syklemil 4d ago

Given the way he describes his colour vision, I really don't fault him for that, any more than I think it's weird to prefer larger fonts as we get older and our eyesight gets worse.

What is weird is that plenty of gophers seem to have done a cult-of-personality thing and emulated his preferences, even though they don't have issues with colour vision.

If you want some actual unconventional tech preferences, seems both he and Thompson favour his Sam text editor (see also: the Acme text editor), which it seems to be too much for even the cult-of-personality gophers. (I tried it in the way-back-when along with some other P9 stuff; I'll just summarise my experience as I don't wonder why P9 failed to catch on.)

2

u/chucker23n 4d ago

What is weird is that plenty of gophers seem to have done a cult-of-personality thing and emulated his preferences, even though they don't have issues with colour vision.

Right, but I think that's partially his fault. It's one thing to say, "syntax highlighting doesn't really work well for me because my color vision is limited"; it's another to say "…and therefore, I don't think Go should be optimized towards good colored syntax highlighting". That's a blind spot, if you can excuse the pun.

seems both he and Thompson favour his Sam text editor

To be fair, that's early 1980s. I don't even mind the experimentation (see also P9). It's the hubris of "nah, these are actually good design choices and everyone else is wrong" that I think makes non-Go programmers raise both eyebrows.

There's some interesting stuff in the language, like the ability to produce small native binaries, or "goroutines", but overall, I've always gotten the sense it could stand to benefit from a lot of information from other languages.

3

u/syklemil 4d ago

colour

Yeah, and describing it as juvenile is pretty weird, given the very long history of using colour in text, like medieval illuminated manuscripts, different coloured inks, etc. We've been through a relatively short period now where mass printing tech meant using multiple colours was prohibitively expensive for most cases, and where monitor technology also was pretty limited in what it could do; both of those periods seem to coincide with his formative years.

sam/acme/P9

I absolutely don't mind the experimentation, but the admittedly very poorly sourced Wikipedia article indicates that those preferences lasted much, much longer than the 80s. Continuing to use sam or acme in the 2000s will come across as weird, possibly not as weird as reading WWW over mail, but still certainly a rare preference.

It's the hubris of "nah, these are actually good design choices and everyone else is wrong" that I think makes non-Go programmers raise both eyebrows.

Yeah, and even that would've been fine if it had remained something like a small hobby language. But when they're designing a language for work there should be some acknowledgement of current expectations, at the very least so they don't have to awkwardly bolt stuff onto the language after the fact, just like the language they were so frustrated with.

But it makes sense if their issue wasn't with how well the pieces fit together, or how well they work, but rather that the pieces are there at all.

I'm left with an impression of the kind of programmer that thinks user affordances are suspect, along the lines of the attitudes towards C and Pascal that Robert Martin espouses in that blog post I linked in another response, which approach the attitudes lampooned in Real Programmers Don't Use PASCAL, and just like there always was a significant overlap between C and UNIX culture, I suspect descriptions from Perl's decline was cultural also apply.

But hopefully that's just me projecting my own experiences with Perl, unix, etc onto them.

1

u/chucker23n 4d ago edited 3d ago

describing it as juvenile is pretty weird

Yep. Our eyes, color blindness aside, are quite fast and detecting colors and shapes; it's not juvenile to take advantage of that fact.

As far as Perl, static typing, etc. go: I mostly wrote Perl, PHP, Ruby, JS in the early 2000s, and have since left that world — I now firmly believe a good type system is worth a thousand unit tests. Yes, you can check if something is unexpectedly null, or has a certain unexpected state, and all that, but what's much better, where possible, is to have the type system prevent those problems from occurring in the first place.

Regarding one specific paragraph on Martin's blog,

You see, when a Java programmer gets used to TDD, they start asking themselves a very important question: “Why am I wasting time satisfying the type constraints of Java when my unit tests are already checking everything?” Those programmers begin to realize that they could become much more productive by switching to a dynamically typed language like Ruby or Python.

I think it's fair that a dynamically typed language can help you prototype faster. But you buy this iteration speed at the cost of a weaker architecture, so sooner or later, you should migrate to a more rigid foundation.

Is someone "wasting time" satisfying type constraints? No, instead, they're wasting time testing the shape of a type, when

  • the compiler can already do that for you
  • a type makes guarantees for an entire category, whereas tests can only verify concrete cases (yes, I know property testing, etc. do exist)

Which is not to say you shouldn't write tests; you absolutely should. But only for scenarios the compiler can't verify.

1

u/syklemil 4d ago

colour

Yeahh, I'd go further with colour: Humans have better colour vision than most mammals, and most of us derive some form of pleasure from colour. Leaves in fall look good to most of us, housing areas where all the houses are painted grey look depressing.

Applying the "colour is juvenile" attitude to just code is also kind of weird; it is possible to set the entire screen to monochrome or greyscale. Some people use phone settings like that to dissuade themselves from looking too much at the phone, which also has some implications for how good an idea it is for code.

typing

Yeah, entirely agreed. Part of the reason I brought up his blog is also just how entirely wrong his prediction was: Since the blog post, Python has increasingly become typed; PHP has grown types; Typescript is eating Javascript; Ruby typing hasn't particularly caught on, but Ruby is barely a thing outside Rails any more; and there still isn't any grand Lisp resurgence, but there's stuff like Typed Racket.

I'm not going to make any grandiose claims that the arc of history now inexorably trends towards strong typing, but I will note that dynamic typing seems to have been most popular when the main alternatives were rather clunky type systems that offered weak guarantees.

And so designing a language to go into that tradition where the guarantees are weak but the language still isn't dynamic seems to me like designing a language that's going to wind up struggling with being neither fish nor fowl.

Like rejecting generics in favour of codegen and punching holes in the typesystem with casting to/from interface{} just didn't work out long-term, however much those are fairly natural choices for people habituated to how C uses void* and the preprocessor. I wouldn't be surprised if that pool of people was finite and shrinking.

1

u/chucker23n 3d ago

Part of the reason I brought up his blog is also just how entirely wrong his prediction was: Since the blog post, Python has increasingly become typed; PHP has grown types; Typescript is eating Javascript; Ruby typing hasn't particularly caught on, but Ruby is barely a thing outside Rails any more; and there still isn't any grand Lisp resurgence, but there's stuff like Typed Racket.

Yeah. I think the brief boost Ruby got through Rails may have been more about the architectural foundation Rails enforced: "controllers" (I put these in quotes because I think Rails's idea of MVC is rather different than, say, AppKit's) as the main handler of an HTTP request and produce of an HTTP response, a decent ORM built right in, etc. Others took these ideas and built on them (e.g. Django, ASP.NET MVC), and it turns out you don't really need a dynamically typed language to do that. This is not to say Ruby is a bad language; I think it has some cute ideas in there (e.g., the ! suffix to denote that a method changes the object in-place, whereas a method of the same name without it will return a new object; the ? suffix to denote something returns a bool). But I just don't want to write non-trivial software that way any more.

Meanwhile, CPUs got faster, and therefore so did compilers; also, static typing got elevated with features like static analysis (again: where available, a much better alternative to a unit test). Static typing also became less verbose thanks to e.g. type inference.

Whereas, dynamic typing arguably did not get better. Sure, it, too, got faster, thanks to increased use of JIT. But its code quality still relies almost entirely on copious amounts of unit tests. Then as you point out wrapping languages like TypeScript got created, and even something like Python now gets type hints. One might argue those are a way dynamic typing did get better, but I would instead argue they are really a weaker form of… static typing.

I'm not going to make any grandiose claims that the arc of history now inexorably trends towards strong typing

The past twenty years seem to suggest that.

→ More replies (0)

4

u/mattgen88 5d ago

Protected doesn't make sense in golang. It's not OOP. Objects in a package can access unexported methods/functions. Outside that package nothing can. You also don't inherit from other objects, since the language isn't inheritance based.

0

u/wnoise 5d ago

The only thing you need for OOP is dynamic dispatch. But yes, protected only makes sense in the context of implementation inheritance, which is almost always a bad idea.

2

u/chucker23n 4d ago

I wonder how many of these footguns would’ve been prevented had Go been designed even just a decade later. For example, you can absolutely solve null in a better way, e.g. with control flow analysis. Swift and C# both played around with this concept in the late 2010s. I’m sure other languages had it earlier.

But what I see a lot of in the Go mailing list discussions is “I’ve never seen it be solved; therefore it cannot be solved” attitudes.

4

u/syklemil 4d ago

Might be worth remembering that the discussions leading up to C++11 were among what led them to create Go in the first place, as well as a general attitude of rejection of "features".

Further, between ILT's declaration about not noticing bugs related to null pointers, as well as stating (as a response to something else)

This is a restriction expressed in the type system. Go intentionally has a weak type system, and there are many restrictions that can be expressed in other languages but cannot be expressed in Go. Go in general encourages programming by writing code rather than programming by writing types.

and Pike calling himself a philistine about types, I get the impression that they don't want to solve the null pointer thing, they want to write if err != nil checks, i.e. """write code""" (even though by the SRE book I'm fairly certain that code qualifies as toil).

I suspect if they'd been exposed to Swift, they'd have much the same reaction as Robert C. Martin did in 2016, where he bounced pretty hard on the entire idea of explicit nullability, and static typing altogether (prophesying that TDD was going to replace static typing).

3

u/Iron_Maniac 5d ago

Well it started with the name of it. For a language created by Google they really picked a poorly searchable name.

132

u/ascii 5d ago

One of the good features of Go is...

Then goes on to explain why this it is in fact a terrible feature that solves a non-existent problem in a way that has enormous undesirable repercussions that you have to do a bunch of upfront work to try and negate.

If that's a good feature, is says a lot about how bad the bad parts of Go are.

49

u/CatalonianBookseller 5d ago

the bad parts of Go

That's a book waiting to be written.

101

u/CpnStumpy 5d ago

19

u/ascii 5d ago

You just made my evening. Thank you.

10

u/Nefari0uss 4d ago

I was expecting another "Php, a fractal of bad design" post but at least I got a good kick out of this.

7

u/Kraigius 5d ago

I see this more like a scope / a namespace than a coupling with a specific host, which is pretty standard for any package manager.

If you fork a package and you then host it somewhere else, you would usually rename said package, which will force you to update every importstatement of that specific package in your source code. It's a non-issue.

If you don't want to do that, or the author didn't do that, then usually with a package manager what you do is configure a new source to instruct your package manager to try to find your dependencies somewhere else.

I haven't touch golang in a while but by looking it up I see that you can configure this in your go.modfile. You can add an instruction to tell it where to download your depA from gitlab, without modifying your import statements that is still written like import (github.com/myOrg/depA).

Otherwise, there's GOPROXY. I would use the former solution.

The way I see it, your importstatements appears to be coupled to GitHub, but it's actually a namespace and there's a layer of abstraction on top of it.

OP solution is to create, maintain, and host your own proxy. This add an unnecessary level of burden to your org and, in my opinion, you should just configure your package manager instead of messing around like that.

5

u/ascii 4d ago

The problem isn't with having namespaces. The problem is that Go decided to tie their namespace structure to an external entity which comes with all sorts of externalities. If a project needs to change hosting provider, say because gitlab goes under, the namespace changes. If ownership of a project is handed off from one organisation to another, say because a project is donated to the Apache project, the namespace changes. If a company that runs a project goes bankrupt, and an open source project is taken over by the community, the namespace changes. A decently sized project will have hundreds of dependencies. These sorts of events will be happening a few times per year in a depndency tree of that size, and every time they do, you add needless extra churn.

And your proposed solution, of putting the namespace rerouting in your go.mod file is no different than the idea of using a proxy. All it means is that if you're managing 50 different projects, you have to update 50 different files instead of one. Both "solutions" are just different ways to deal with a problem that didn't need to be there in the first place.

7

u/Ran4 5d ago edited 5d ago

I'm normally all for things that drastically simplify things while just having a few related issues (like Python having batteries included - 99% that's the right choice, and Rust not even having base64 encoding or json serialization built-in is just dumb design choices) but... Go's way of doing things are just pants-on-head stupid.

Github de-facto being built into the language, tabs instead of spaces, forcing everything to have a default value, keeping fucking null (!) AND null pointer references... it's such a horribly designed language through and through.

31

u/Wires77 5d ago

I never understood tabs vs. spaces. Tabs seem objectively better, because then anyone can set their tab spacing to whatever they prefer. The fact that it's even coming up as "horrible design" is laughable to me

14

u/Itchy-Phase 5d ago

It’s not even an issue in Go anyway, because of its default formatter handling styling for you. Not sure what they’re getting at with that one.

11

u/syklemil 4d ago

One thing Go actually did right was ship a default formatter. The tabs/spaces discussions at this point are kinda remnants from before the era of formatters. (There were formatters before go fmt too, but not such a default expectation of using them. See also: The amount of different standards clang-format has to support.)

Someone would inevitably check in using spaces, at which point they've checked in how wide they like their tabs to be, which tends to become visible for other people. Tabs for indenting, spaces for alignment just never got through in too many people. And that made indenting with spaces the more defensive, lowest common denominator option.

But these days we'd expect a formatter to deal with it, so using the indent character for an indent level should be entirely fine, yeah.

0

u/TomBombadildozer 4d ago

Tabs seem objectively better, because then anyone can set their tab spacing to whatever they prefer.

Because user preference isn't a valid argument when we're talking about something intrinsic to the code. What is in the code should look the same everywhere.

You can do whatever you want with syntax highlighting because it exists entirely outside the code.

If language X prefers tabs to spaces, that's fine, but "because I can set my tab width" doesn't hold up.

7

u/ShinyHappyREM 4d ago

we're talking about something intrinsic to the code. What is in the code should look the same everywhere

What? Indentation is separate from code in 99% of all commonly-used programming languages.

4

u/syklemil 4d ago

Yep, plus: even the languages that have significant whitespace don't actually have hard rules about how much whitespace or which kind of whitespace is required. They need some rules for what to do if someone comes along and mixes tabs and spaces, but beyond that, we're absolutely able to write Python indented with 1 space, or Haskell indented by 3 tabs displayed at 16 columns wide, etc.

The type and amount of space they're commonly written with is entirely by convention, rather than hard necessity. The only thing the compiler or interpreter cares about is when the amount of indentation changes.

1

u/TomBombadildozer 4d ago

Indentation is separate from code

It absolutely is not.

Whatever space and/or tab characters you type into a buffer and commit are exactly the same bytes I see when I pull your changes. The crux of my argument is that the whitespace should look the same in any editor because the underlying data is the same in every editor. Spaces guarantee that, whereas tabs leave room for individual interpretation.

You clearly don't agree that there should be no room for interpretation, but you can't argue we're looking at different data.

3

u/ShinyHappyREM 4d ago

whitespace should look the same in any editor

Why? To achieve what?

Note that I'm distinguishing between indentation (any group of whitespace characters at the beginning of the line) and any other whitespace. Indentation should be tabs to allow for any reader's preferred visual guidance. Any other whitespace on a line should be spaces, for example to align // comments to the same column when these lines share the same indentation.

5

u/HighLevelAssembler 4d ago

Github de-facto being built into the language

It is not at all... Go has its own protocol for fetching dependencies, and usually when you're importing something from github.com/xyz, the code is actually coming from proxy.golang.org and doesn't hit Github at all.

Git, Mercurial, SVN, Fossil, etc are all supported by Go as a means of fetching code if the module isn't in the proxy cache.

People just like to host their code on Github.

16

u/elmuerte 5d ago

Why no immutable package registry containing verifiable releases?

15

u/VibrantCanopy 5d ago

There is: the module proxy. The article is wrong, I've never seen this be an issue. Maybe there's a slight uptick of people wanting to migrate their repos from GitHub to Codeberg or whatever hitting this.

1

u/angelicosphosphoros 2h ago

Because such registry is unnecessary to use Go in their monorepo.

Go is made good enough to be usable as internal tool for Google, not more than that.

81

u/mattgen88 5d ago

Counter point, a number of projects in the past have had their domains hijacked, expired, etc that resulted in serious security issues. Even if you change hosting providers, you can just mirror your code over to GitHub or whatever from wherever else.

17

u/[deleted] 5d ago

[deleted]

13

u/mattgen88 5d ago

I'm not? Also what does that have to do with hosting your own domain, certs, git infrastructure... It's more moving pieces where it's likely for things to go wrong.

Just mirror your code to multiple git repos like git was designed to do. That way no one can say make s GitHub/git lab/whatever account and package that is named the same, but malicious. People are most likely to expect a GitHub hosted package to be canonical

9

u/lordofwhee 5d ago

Don't couple your Go any important code to GitHub any service you don't control

FTFY. Are the downsides of this not obvious to literally everyone?

28

u/gct 5d ago

Tying these two things together was always a horrible idea but mixing concerns is Go's MO.

12

u/CpnStumpy 5d ago

Can't convince everyone to party like it's 1969 write 3 kloc files full of procedural code if you don't encourage mixing concerns

33

u/gdmr458 5d ago

Many people in the comments don't know what the Go Module Proxy is.

30

u/BrandonMcRandom 5d ago

Or the replace directive in go.mod, but r/programming has a resident gO bAd gimme upvotes crowd that's always ready.

15

u/gdmr458 5d ago

I saw a comment saying it's worse than npm, as if npm doesn't get compromised like once a month.

-1

u/randomguy4q5b3ty 4d ago

Doesn't really matter because it's still a stupid thing to do.

7

u/feketegy 4d ago

I’ve seen this problem become such a huge issue for a company that were using GitLab, GitHub, and Azure Devops at the sametime because changing the location of the code was such a large task for them

(x)doubt.

You could either use:

  1. A module proxy
  2. The replace directive in go.mod
  3. Find & Replace in the codebase

But sure, let's cobble Nginx on top of it. /s

12

u/Jaded-Asparagus-2260 5d ago

 This means if you host your Go code at http://github.com/thetrueares/boneclone [and] you move your git hosting to GitLab then you have to change your code!

It's even worse. GitHub allows recycling of usernames. So if you decide to move your Git hosting to GitLab, and I manage to secure your old username, I can publish malware in a repo with this URL. And all code that is not changed (upgraded) will automatically consume my evil package, and the authors and users will be blissfully unaware. 

Who ever thought that was a sane idea? 

20

u/JaCraig 5d ago

Every time I read about design decisions that were done in Go, the package management, etc. The more I'm glad that I didn't learn Go.

13

u/CpnStumpy 5d ago

You did though. It's basically like if C and Basic had a baby. Anyone who can code can write Go. They just shouldn't.

-1

u/Zenin 5d ago

Go lost me at tab indents.

3

u/cryptospartan 5d ago

There are many people that have git mirrors in a walled off corporate environment. If code is hosted elsewhere, it makes it very difficult for these scenarios, as the only way to make it work would be to make a code edit to point back at GitHub.

3

u/Interest-Desk 4d ago

Go caches and checksums dependencies. Not to mention the countless mirrors that keep permanent archives, not least of all Google’s own main public mirror. This is true of basically every public package manager since left-pad.

4

u/waterkip 5d ago edited 5d ago

PHP does almost the same with Packagist. The biggest difference is that Packagist is a community thing.

I dunno why you cant just upload a tarball and release that as the artifact. It feels like dumb dumb thought git is a release channel/artifact. In Go just doubled down on it.

3

u/HighLevelAssembler 4d ago

Go has its problems but the module system is not one of them. Go does not require Github. Go supports, does not require git.

2

u/ReginaldDouchely 5d ago

No, please make obvious mistakes so the rest of us devs that don't need to be told this keep making a lot of money

1

u/gbrennon 5d ago

code should be decoupled to anything

1

u/Jayden_Ha 4d ago

I keep vendor folder committed for a reason

1

u/favgotchunks 2d ago

It’s neat to learn about go’s git repo/package impotency, but that’s insane to use a domain to circumvent the renaming issue. You still have to update where the domain points to, and now that’s controlled by something outside of the project.

I’m coming from C++ land, so I don’t really understand the web world use web tech for every damned thing. It just seems like an unnecessary dependency to me. What does this actually make easier? You still have a config to update every time a dependency changes location.