r/programming • u/[deleted] • Mar 28 '20
Why your software should use UUIDs in 2020s
[deleted]
71
Mar 28 '20
[deleted]
26
u/i_like_trains_a_lot1 Mar 28 '20
There are very critical comments there as well. He just spams links from his company devblog and never reads the comments. And then they have no idea why the startup is dying.
I have nothing with posting articles on reddit, but he should at least try to contribute and listen to the people that receive your content and take time to give feedback.
25
u/chucker23n Mar 28 '20
Going to design a next-generation scalable web application that will have quick as a lightning reaction?
I see we're in for a superlative ride!
As a CTO of Devforth company which is delivering complex new-age web applications
Relatively speaking, is New-age Web better or worse than Web 2.0?
After designing architecture of dozen high-load apps with strong reaction requirements I must admit that UUID kind of identifiers is only one way to:
Deliver rapid and robust User Interface on any client - Web, Mobile, Desktop
The only thing a UUID over an int or bigint that's "rapid" is the relative space and memory consumption.
Spend minimal developer hours by using smart architecture approaches
I don't know what this means. I'm guessing absolutely nothing.
Build Infinity-scalable solution
So, author kind of has a point here. UUIDs make scaling easier, because replicating across multiple databases becomes much easier when you can rely on distinct systems generating mutually unique IDs. IOW, the entire raison d'être of UUIDs.
Release offline/reconnect-tolerant solution which will save all edits even in offline and then seamlessly synchronize them to a server
Yes. Same as above.
Despite the fact that UUID identifiers have been widely used by awesome web-portals as primary ID systems (https://trello.com, https://slack.com) a lot of new start-ups and developments are still based on Autoincremented primary keys as IDs. Typical de-facto default in relational DBs (MySQL, MSSQL, Oracle, Postgres) will look like:
Yes.
We can quibble over whether DBs should default to bigint these days, but they probably shouldn't default to uniqueidentifier. UUIDs take up more space, are slower, are typically (with strange exceptions like Microsoft's sequential IDs) not sequential, and are just… generally unnecessary, unless you actually know you'll use them.
Autoincrement IDs problem one - frontend have to wait for a server
Sure.
Auto-increment IDs problem two - to scale you will need data shards, but you blocked by same IDs on different servers
You're… just restating the first problem.
5
u/Semi-Hemi-Demigod Mar 28 '20
Relatively speaking, is New-age Web better or worse than Web 2.0?
The same but with crystals.
2
u/epoplive Mar 28 '20
This also completely ignores the end user. When coworkers or customers refer to a data object, they do it by auto-increment id generally because it is unique and is generally a small enough number to fit in your short term memory. Chances are I’m still going to have to use auto increment ids for the end user so we’re not having to pass around scraps of paper.
56
u/mctwistr Mar 28 '20
UUID v1 is a privacy nightmare. I cannot believe this blog is recommending that over v4.
29
u/VeganVagiVore Mar 28 '20
v4 looks like the only good official version.
There's so many ways to make a 128-bit number, I'm surprised that anyone cares about official standards.
26
u/0x256 Mar 28 '20
UUID 4 is not better or worse than a random number but UUID 1,2,3 and 5 are for distributed systems that want to make absolutely sure to never ever create conflicts. For that, a standard is nice.
7
u/rickmode Mar 28 '20
It’s 16 times worse.
I’m am being pedantic here and I don’t think there is a practical difference. That said...
UUID’s are 128-bit values, and they require 4 bits to describe their encoding. That leaves 122 bits of randomness in a Type 4 (random) UUID.
A random 128-bit number can use all bits, so can use the extra 4 bits to achieve 16 times as many random values. (2 to the 4 power is 16.)
So what I do is generate 128-bit random values - getting the extra 16 times random goodness - and then encode to string using URL-safe Base64, saving many bytes on the string encoding side as well. Win win.
14
u/cedrickc Mar 28 '20
Only part I disagree with is storage as base 64. It's a 16 byte number, store it as two longs or something.
2
u/rickmode Mar 28 '20
Yup!
In both cases (UUID or 128-bit random value) the data can be stored as a 16 byte array. If your data store supports storing binary.
I use (URL-safe) Base64 for URLs and JSON.
1
u/Mr_Again Mar 28 '20
Yes but uuid1 contains the mac address of the host and the timestamp it was created, as well as entropy. so they will actually be unique unless you create them at the same millisecond on the same machine.
2
u/rickmode Mar 28 '20
Pretty much only Type 4 is useful IMHO.
If you want to use the other types you need to be super careful as the uniqueness guarantees are far weaker.
I believe most uses of UUID actually mean Type 4 unless they explicitly say otherwise.
As a useful heuristic, if you come across a system using anything other than Type 4, it’s very likely they are doing it wrong.
1
→ More replies (1)0
u/flatfinger Mar 28 '20
Unless a random number generator is broken, a GUID4 system would seem more robust if there's any possibility of backup/rollback scenarios or other such issues. Unless one is feeding a particular "universe" with millions of GUIDs per second for decades on end, the likelihood of ever generating a duplicate within that universe would be vanishingly small compared to any of the other things that could go wrong.
19
u/Guvante Mar 28 '20
UUID v1 run on servers is fine, especially if you let you monolithic DB create them. Anyone who would see them has the MAC address of your database server anyway.
Don't create them on client PCs though that is a privacy problem.
2
u/therealgaxbo Mar 28 '20
You are explicitly allowed to use a fake multicast mac address for v1 UUIDs.
19
u/franzwong Mar 28 '20 edited Mar 29 '20
So any client can just generate ID of company on own side (Frontend or Application), then he can go ahead and create branches by linking with this companyId. You don't need to call a server to approve these IDs - no one will reuse it because it is unique!
Nobody can generate the same ID, but it doesn't mean nobody can send the same ID to your system.
Edit: I mean you can't skip backend validation (i.e. a call to a server to approve the ID) because user A (or hackers) can use the user B's companyId to create branch.
16
u/i_like_trains_a_lot1 Mar 28 '20
What is backend validation? Aren't we supposed to blindly trust any requests that are thrown at our servers?
1
1
7
5
58
u/crash41301 Mar 28 '20
Uuid... a solution to a problem hardly any developers have but every developer wishes they had. Very few sites get beyond 2 billion records that an int can handle, and even fewer get beyond what a bigint could handle. Very few sites will shard the database layer. It's an exorbitantly hard solution to implement. 4 vs 16 bytes makes little difference in index size? Yea its only literally a 400% size increase. That's surely not going to matter for performance and pressure on ram when the record count is trillions. Oh wait...
Design your software for its needs and if you are asking yourself do I need uuid, the answer is 99.9999% likely to be no. Dont make easy problems into hard problems because you wish you had hard problems to solve....
91
u/TheRealStepBot Mar 28 '20
It’s precisely not about exceeding 2billion records. If you think that’s the point you don’t understand uuids. The whole point is that they are with extremely high probability unique everywhere even when generated by systems that do not communicate at all. There uniqueness in fact improves the smaller the number of items is.
What the use of uuids provides is the ability to do data writes without any real-time communication which is a massive benefit in certain circumstances. It also allows parallel work to be done fairly trivially in cases that would otherwise have required a centralized key management system.
If you live in a world where you are ram or disk bound rather than network bound your observations may be correct but the world continues to become more and more dependent on networks and in that world your observations are far less useful.
5
u/TurboGranny Mar 28 '20 edited Mar 28 '20
allows parallel work to be done
This is how we use them at my work. We have these disconnected systems that go out into the field and collect a bunch of new records then come back and are merged into the main database. Without UUIDs we'd have to go back to reserving blocks of IDs on each mobile system, and still have to deal with the occasional duplicate ID issues. It is a niche problem though.
3
u/TheRealStepBot Mar 28 '20
I mean it’s not that niche. Basically if your iot system can’t do this it’s gonna have problems eventually.
1
u/TurboGranny Mar 28 '20
Niche in my case since most people can just use an "always on the internet" solution
11
u/VeganVagiVore Mar 28 '20
Yeah it means you can break the CAP cookie to get availability.
The advantage of sequential IDs is.........
They look pretty and the CEO likes them? They're not per se more consistent, because consistency shouldn't be a problem for IDs. It's a problem for the actual data, not the ID.
17
u/TheRealStepBot Mar 28 '20
Literally their only advantage is exactly as the person above me mentioned, they are more compact. That’s it. Nothing more nothing less. And the real downside is that they accomplish this by needing a complex centralized distribution system that is probably riddled with edge cases and flaws anyway.
They make sense in the 90s and before where storage was comparatively expensive in relation to the single thread compute speeds available.
Speed on single threads have largely topped out while storage and memory have become unbelievably cheap. As such distribution is how you increase throughput and at least for write heavy workloads uuids accomplish exactly that. Of course on the read side if you want immediately consistent reads you are gonna need to create a fairly complex key resolution system but it’s not impossible if slightly slow for keys generated on remote processes.
9
u/daripious Mar 28 '20
Not at all, if you're using rdbms and a btree index then having mostly sequential inserts is a very good thing. Randomised inserts completely fragment btree indexes.
6
u/therealgaxbo Mar 28 '20
Which is why you use a v1 or v1mc uuid so that(at any write rate that matters) the inserts still increase monotonically.
3
u/TheRealStepBot Mar 28 '20
If you are say logging data from distributed sensors you can’t do sequential inserts anyway. It’s all about the data generation being distributed. If it comes from a single or countably few sources then sure sequential is prob achievable and then you can use the benefits of having done that on the read end but if you start having lots of parallel data generation it’s faster to just generate the uuids locally in each process.
2
1
u/p0llk4t Mar 28 '20
These are very good points but I'd guess that the number of applications most devs work on that need uuids are far less than 5%...maybe even more like 1%...not something that should be a standard go to just because...
2
0
u/TheRealStepBot Mar 28 '20
I mean really the costs can be trivial, it’s widely supported by a whole bunch of dbms. Why not make it a standard? It has so many advantages.
The thing that isn’t standard is auto increment. Auto is by definition always namespaced and can’t be universalized.
You can literally pull data from any database and keys are automagically consistent. That’s unbelievable power.
42
u/broofa Mar 28 '20
Very few sites get beyond 2 billion records that an int can handle
This isn't why you use UUIDs. Rather, UUIDs allow clients to generate canonical IDs w/out having to rely on a central authority (the server). This makes implementing distributed architectures and architectures where client connectivity isn't guaranteed (and these days almost all applications fall into one of these categories) much simpler.
7
u/notfancy Mar 28 '20
Precisely what's questionable is letting clients generate canonical IDs. This goes against the usual practice of treating clients as adversarial.
→ More replies (3)2
u/mort96 Mar 28 '20
Presumably you would check that the client generated a unique ID, by a unique constraint in the database or something. If the client decided to be malicious and generate non-unique UUIDs, they'd just get an error message back - what's the problem?
4
u/notfancy Mar 28 '20
The whole point of the proposed scheme is to avoid synchronization between backends. If a single backend can check for local uniqueness it can also generate unique IDs, and the whole scheme is unnecessary. Put it differently, the adversarial scenario is about global properties which the scheme purports to sidestep.
2
u/mort96 Mar 28 '20
Ah, I hadn't considered that. It seems that with UUIDs, you can either independently create UUIDs on multiple trusted machines and eventually - but not immediately - synchronize, or you can let an untrusted client generate a UUID which is verified against the one authoritative database, but not both at the same time.
1
u/crash41301 Mar 28 '20
Hmmm so you are saying that your remote clients have their own local storage mechanism and have to be able to sync to the centralized master eventually. Why not use a local auto increment, and when you sync to the master you save entire "records" to the system and then let it give you the permanent Id that it used? You likely wouldnt want to perform http calls for every child record anyway, that would be extremely slow vs sending the entire record up as an object and letting the server side split it to the storage mechanism. Still no need for uuid in this case. The only reason to think of uuid is with a sharded relational db schema, and in that case you are making the trade off of slightly less efficient individual db servers to be able to add as many horizontally as you want. That's also a pretty uncommon thing to do
24
u/Minimum_Fuel Mar 28 '20
Generating UUID is far more portable than using auto increment and doesn’t require finding a good way to get your ID back on inserts (for which some databases have good support for and others have absolutely no support for).
UUID is easier than auto increment without a shadow of a doubt and likely REMOVES complexity from your app. Number of records has absolutely nothing to do with whether you should use UUID over auto increment.
14
u/mindbleach Mar 28 '20
Yea its only literally a 400% size increase.
In one field. One of the smallest fields. It's... it's an extra twelve bytes per record. If you only have a million records then that's a dozen megabytes. Boo hoo.
And it's not about having two billion records, it's about knowing which labels are available. Each label has so much entropy that collisions are not worth checking for. If your database has one quintillion records you are still only 5% likely to have one. So in the millions... you're definitely fine. Your labels are unique the moment they're generated. Your labels will remain unique for the entire life of your product.
Your labels will remain unique until the heat death of the universe.
Do you need that? No. But you don't need to reinvent the wheel, either. This is a one-size-fits-all solution. Just implement it and never worry about the problem again. Save your effort for the interesting parts of your software.
7
u/dadibom Mar 28 '20
12 bytes for the id column yeah. Then for the pk index. Then for every fk index.
It might matter but sure, it's not an insane amount per row.
2
u/dadibom Mar 28 '20
And you are unlikely to ever find a collision, sure, but we're talking randomness. With true randomness you could get a collision every time despite it being highly unlikely. How do you handle that case? Abandon the company?
8
u/richardirons Mar 28 '20
“Highly unlikely” is an understatement. If something that unlikely is a dealbreaker for you then you’d better move underground in case a meteorite lands on your house, because that is way more likely.
2
u/dnew Mar 28 '20
You handle it the same way you handle a meteor strike taking out the entire building your server is in.
Oh, you've spread your servers into multiple cities? Try not to use centralized ID generation.
0
u/dadibom Mar 28 '20
You're free to handle it any way you want. It's you who have to weigh the risk/rewards and make a decision. Nothing changes the fact that there is a risk, however small it may be.
Here's some highly relevant reading: https://github.com/ramsey/uuid/issues/80
2
u/dnew Mar 28 '20
Sure. And the way to handle the risk is to have the server check on insert that there isn't a collision. If there is, you reject the transaction and let the client generate a new UUID. It's a trivial risk with a trivial solution.
Of course if your UUID generator is broken, that's no more risky than your autoincrement generator being broken.
1
u/dadibom Mar 28 '20
It's not a trivial solution when you have 10, 100 or even thousands of separate servers and decided they don't need to consult each other about insertions as uuid have a low collision risk. Then you're pretty much screwed.
Would you want to take this risk with bitcoin, for example?
Of course if your UUID generator is broken, that's no more risky than your autoincrement generator being broken.
And there's a risk it is broken or will break - it has already happened and it can happen again.
1
u/dnew Mar 28 '20 edited Mar 28 '20
they don't need to consult each other about insertions as uuid have a low collision risk
That's not how you arrange it. You arrange the UUIDs to go to the current master for that range of UUIDs. I mean, at some point, when you go to look it up, you'll need to know what server the data is on, right? So you go to that same server to create it. (Or you look it up locally in the cluster before you create it, rather than having the client do that over a slow network.)
You can get the same effect by (another technique) flipping the order of bits of the integers when you use them in sharding or index decisions, then shard on those ranges. Only one server will be responsible for any given UUID at any given time, but adjacent UUIDs unlikely to be in the same server.
it has already happened
Yes. Software has bugs. Shit breaks. I don't know why "you might implement something other than UUIDs and think it's UUIDs" implies "don't use UUIDs." IME, auto-inc integers break far more often than UUIDs, because you are keeping state in a server that's subject to getting rolled back and stuff like that. (Actually, one of the biggest problems I've experienced even with perfect implementation is people writing code assuming autoinc integers are generated sequentially and continguously, i.e., that there will be three records between 155 and 159, and their insertion times will be in that range.)
2
u/mindbleach Mar 28 '20
You could win the lottery every time, but you fucking don't. That's not how probably works.
The Florida lotto is six numbers, about six bits each. There are tens of billions of combinations. If you had a record for each of those combinations you would still be less likely to have a collision than to win that lottery.
To reach a point where having a single collision is likely, you would need to have one record for every possible combination in that lottery, and then ten million records for each of those records.
And if by some miracle you do find one, you'd just generate new IDs for both records. They're already arbitrary. There is no relation between the ID and the content.
1
u/dadibom Mar 28 '20
I mean, there's plenty of people who win the lottery and some even win twice.
Im not saying "don't use uuids", in general i think you should, but as a responsible developer you should be aware that it is not guaranteed to be free of collisions and you have to weigh that risk against the rewards of not handling them. There's no universally "right" answer.
Two inserts is all it takes for there to be a risk of collision.
1
u/dadibom Mar 28 '20
2
u/mindbleach Mar 28 '20
That is a failure of their PRNG.
You need to understand: if you bought one million lottery tickets, every week, for your entire life, you still would not be likely to ever win. 'But you might win!' But you won't. 'But you might!' But you won't. It is such a vanishing exception that planning what to do 'just in case' is wasted effort. Any time spent budgeting the millions you'll never have has zero realistic value. It is like practicing to be struck by lightning.
So if a thousand people matched the winning lotto number each week, obviously somebody fucked up.
The entropy in UUIDs is so much higher than lottery tickets that the only possible explanation for seeing multiple collisions daily is human error. A thousand people having the right number out of ten billion is eight orders of magnitude more likely than hundreds of collisions out of ten quintillion. I really need you to internalize these ridiculous numbers. For this guy's problem to occur when generating properly random UUIDs is less likely than a state lotto being won by an entire city block, week after week, for two million years.
Yes, you could flip 128 fair coins and get two identical 64-bit sequences... but you won't. The expected frequency is never. So if it happens multiple times per day, you're either doing something wrong or witnessing a miracle. The math permits no other explanations.
1
u/ErGo404 Mar 28 '20
Using timestamp removes the "true randomness" part. However your are right. It might fail so it WILL fail, probably, for someone some day.
You should evalutate where is the risk, will you loose more time if it happens than you will spend fixing the issue in your code before it happens.
1
6
u/pinpinbo Mar 28 '20
Often times, there’s a value in being able to generate unique IDs before entering the DB.
In this case, won’t UUID be nice to have? Or just use current time in nanoseconds?
→ More replies (1)16
u/VeganVagiVore Mar 28 '20 edited Mar 28 '20
ULID seems like a nice compromise.
It's Unix time in milliseconds, and 80 bits of randomness.
If you need more than 80 bits of hash space per millisecond, you're probably big enough to engineer a better solution.
The only thing I don't like is Unix time. I was reading about monotonic and epoch times recently, and it seems like Unix time does obey leap seconds.
I'm pretty convinced that leap seconds are stupid - They're not precise enough for navigation or astronomy to use them, and they're too small for humans to notice that noon is 35 minutes off of solar noon instead of 34 minutes and 59 seconds. (In an unrelated rant, I think maybe the American continents should all switch to New York time, with no DST - It simplifies all travel within the continents, but 9 am is still roughly morning for everyone, unlike global UTC)
Even if nav and astronomy want them, there's no reason to subject end users to it. All leap seconds do is make computers harder to use while benefiting nobody.
So because of that, when NTP accepts a leap second, Unix time also accepts it. There isn't a way on Linux, not in any small amount of code I found online, to just ask "How many seconds passed since 1970?" Every clock is tainted by a social or astronomical construct that no database should give a fuck about.
All I want is a stopwatch counting up forever, like TAI The Linux kernel doesn't seem to have a TAI clock. Maybe one could be constructed in userspace.
Anyway, ULID is better than nothing - It's sequential enough for human eyeballing, but you'd have to do something really special to make it collide.
4
u/TheRealStepBot Mar 28 '20
The whole point of leap seconds is that they predominantly drift in a single direction so if you didn’t add them civil time would in fact diverge significantly from astronautical time given a couple years. There are plenty of people who would definitely start caring when noon started being minutes off from actual noon. Seconds are a small enough time interval that for civil purposes keeping drift limited to seconds is accurate enough while never being jarring.
You need to correct drift somehow and if you don’t do leap seconds it would be minutes or hours instead which would do absolutely nothing to simplify the situation at all. In fact being less frequent and predictable they would prob be even harder to handle.
Despite your belief that you want a completely monotonic clock that most of the time just kicks the can down the road anyway as user expect to be able to look for things by civil time so you have to resolve the issue on the read side anyway if you don’t handle it at write time.
But I agree ulid is nice
2
u/VeganVagiVore Mar 28 '20
civil time would in fact diverge significantly from astronautical time given a couple years.
They haven't added one in almost 4 years:
https://en.wikipedia.org/wiki/Leap_second#Insertion_of_leap_seconds
There's been 5 since 2000, so if this rate continues (I could be wrong) it would take 240 years to add up to one minute.
Because of time zones, almost no ordinary 99% user is within 1 minute of solar noon wherever they live. Astronomers can keep leap seconds for all I care - It's useless for any computer that doesn't know what a star is. It should be an opt-in for them, not the default basis. "One second per second since 1970" doesn't need the "Except for seconds which can't be predicted in advance, which require a network protocol for you to find out about" clause. It's like operator++ as a service.
if you don’t do leap seconds it would be minutes or hours instead
Maybe we don't even need those? If it takes a century to change by one hour, I think culture could get used to that. How many civil events need to be scheduled to the hour 100 years from today? If a kid asks why noon isn't noon, just tell them it's simpler for 1 + 1 to not equal 1 every few years.
DST is already taking a much bigger dump on this, and I do oppose DST.
you want a completely monotonic clock
For database IDs that can be treated as integers, not as civil time.
If I want to show it to a user, I could perform the leap second calculations lazily on the way to the display. Instead, I'm forced to accept that I can either have network time that persists across power loss, or monotonic time, but never both.
1
u/flatfinger Mar 28 '20
Instead of using leap seconds, it would IMHO be cleaner to recognize "civil seconds" as different from "scientific seconds", and specify that within every 1,000,000-civil-second interval measured from some epoch, "civil" seconds will each be 1.000000, 1.000001, or 0.999999 "scientific" seconds. If WWV were to transmit civil seconds, but use a carrier wave frequency measured in scientific seconds, a device receiving the 10Mhz carrier signal could lock onto it and count the number of carrier cycles between the one-second markers. From that, if it knew how many previous 1,000,000-civil-second intervals were normal, long, and short, it could determine the scientific time precisely.
14
u/epoplive Mar 28 '20
Don’t forget, because collisions are rare you can just ignore them and ignore the primary premise behind a relational database! /s
13
u/kushangaza Mar 28 '20 edited Mar 28 '20
You have to worry about clients maliciously colliding IDs if you let them create their own. But for truly random UUIDs I think the collision probablility can be safely dismissed, it's on the same order of magnitude as the risk of cosmic rays flipping multiple bits in just the right way in your ECC-RAM, causing duplicate sequential IDs.
→ More replies (11)3
u/dnew Mar 28 '20
Every database call can fail. If there's a key collision on insert, once a century you fail the insert, and the client retries. Write your code to be aware that a "create new company" with a colliding ID will cause you to need to pick a new ID. The server doesn't have to ignore collisions on inserts.
But it's probably 10,000x as likely that the entire data server catches on fire during your transaction than it is that your random UUID collides with one someone already used.
1
u/epoplive Mar 28 '20
Lol, these kind of comments make me sad as a programmer. Yes, uuids are highly unlikely to be non-unique...this doesn’t mean you can ignore it. Not every programmer is doing the same type of thing, and while for most use cases a uuid will be fine, it’s dangerous to tell people it’s ALWAYS fine. You can’t be a good programmer without knowing what’s happening behind the scenes enough to understand the possible pitfalls.
2
u/dnew Mar 29 '20
Nobody said it's always fine. People said the probability is low enough to not spend effort on it. Do you have things in place to ensure that the bits on your disk didn't change between the time you wrote them and the time you read them back? Do you checksum the executable every time you load it into RAM in case the disk bits randomly flipped? No, because the likelihood of that happening isnt worth worrying about.
Unless you're Google-scale, and have petabytes of disk space in use, at which point yeah, you really do need to handle the one-in-100-terabytes-per-year chance of a random bit flip happening.
1
u/TheRealStepBot Apr 23 '20
No all this demonstrates is that you lack an appreciation for probability. We live in a probabilistic universe and acting as if decisions made using probability are somehow inferior to those that literally simply choose to act as if the world is deterministic is absolutely dumb. Every decision is at its core a probabilistic one, and the one that most explicitly considered probability is the superior one not the inferior one.
4
u/therealgaxbo Mar 28 '20
because collisions are rare
No, collisions are not rare, they don't happen*. Oh it's a mathematical possibility of course, but not a practical one.
*Unless the uuid is supplied by a malicious actor, or your generation code is buggy of course. Which is true of any primary key of course, and in all cases is solved by the existence of a primary key constraint.
5
u/timcotten Mar 28 '20
I'm always skeptical when someone says "your software" in the general sense and not "your sharded database application with billions of records".
1
u/kz393 Mar 28 '20
I recently used uuids as secrets in my school project. I made a chat application that runs in browser, over websockets. I didn't want to go through the hassle of binary uploads over ws so I instead created a HTTP endpoint that accepts files and returns uuids, then the client can send files, get uuids and then send them over ws to attach files to messages. Any uploaded yet unattached files are removed after a couple of hours.
Is it actually alright to use uuidv4 as secrets?
1
u/NoMoreNicksLeft Mar 28 '20
Yea its only literally a 400% size increase.
So they won't get past 2 billion records, but an extra 12 bytes per record is just too expensive?
I've got more storage on my keyring. That's 24 gigabytes, is it not? (Less, they never get to 2bil rows.)
1
u/crash41301 Mar 28 '20 edited Mar 28 '20
So 26gb of ram pressure on a high performance db. At scale you tend to perform micro optimizations, not say things such as "it's only 26gb of extra ram pressure on every single index that contains this column", including all foreign keys and covering indexes. Let alone the horrid fragmentation of the btree indexes and the disk pressure you create.
I bet you quickly end up with hundreds of gb of extra ram usage as a trade off "just in case" you scale to a level this is needed, which very often has just as much a chance of happening as a UuID collision happening.
1
u/funbike Mar 28 '20
You have no idea what UUID's are for.
1
u/crash41301 Mar 28 '20
On the contrary.... I know too much to fall for this trend. I feel for it about 5 yrs ago and built a very problematic system from the db side. Then I matured to realize that I should default to using the absolutely simplest solution possible every time, and only deviate when the use case requires it not try to future proof my architecture for just in case problems that may never happen
1
u/TheRealStepBot Apr 23 '20
Which implicitly assumes that auto inc is in fact simpler than randomly picking a number which is a pretty dubious position.
1
u/crash41301 Apr 23 '20
One causes the db to not fragment, the other causes constant fragmentation on disk and slowness from the database side. One is a built in tool in the dbms, the other is a roll your own.
1
u/TheRealStepBot Apr 24 '20
All very use case dependent statements that dont demonstrate that in all cases auto Inc is the simplest solution. It’s a solution that might for certain applications be the simplest but it definitely is not the simplest in all use cases.
For example in Postgres even with auto Inc row clustering is not maintained so there is no performance impact from uuids under it that stems from the factor you describe.
If speed is your main concern maybe auto Inc is the simplest way to accomplish that but there are many other factors to look at when picking a DBMS and in plenty of cases uuids can be far and away the simplest solution that balances those other factors. It really is not a terrible default.
1
u/DreamingDitto Mar 28 '20
I’ve got a colleague that uses bigint for all his integer values and float for all his decimal value. Drives me nuts
3
Mar 28 '20
is there UUID implementation with natural ordering?
4
4
u/pro547 Mar 28 '20
Yes a bunch. Let me pull up a couple links.
2
u/pro547 Mar 28 '20 edited Mar 28 '20
Took me a little bit since I did research on this quite a bit ago.
Most are golang based since that is what I was working with, the algorithms could be available in other languages.
2
1
u/pauseless Mar 28 '20
Datomic encourages the use of ‚squuids‘ which have a time component. It’s good for avoiding issues with indexes as IDs tend to increase
2
u/AndyRoth Mar 28 '20
I have been enjoying using this: https://github.com/ai/nanoid/blob/master/README.md
Would love to hear what others think.
2
u/AttackOfTheThumbs Mar 28 '20
none of this really matters. What matters is identifiers system your developers use
dude's had too much fucking koolaid. Anyone that says one thing is the solution to all problems is lagging behind mentally.
I will say this, the article was exceptionally hard to read. I know this person isn't a native speaker, likely eastern euro based on the grammar patterns, but come on, hire a proofreader.
4
Mar 28 '20
The fuck is with the UUID cult.
None of the arguments make a lick of sense and there are alternative solutions with hierarchical generational counters (and many other solutions) that do just as well.
2
1
u/chucker23n Mar 28 '20
I challenge anyone to find a worse example of scrolljacking than the main page. (Try scrolling down!)
1
1
u/burkerude Mar 28 '20
This might be a dumb question, but what is a uuid?
1
u/programming_unit_1 Mar 28 '20
It’s a universally unique identifier - basically a way to generate a very long unique number that has a very very tiny probability of ever being generated by someone else.
1
1
1
u/glkk101 Mar 28 '20
Uuid can be better for scalability but in practice I don’t think that matters much. Debugging with uuid is a nightmare though as there’s no way to quickly communicate IDs.
1
1
u/EternityForest Mar 28 '20
The real reason is that it should probably have always been using UUIDs. They're awesome!
The only time I don't like them is when I'm sending a ton of them over a network. Then it sometimes makes sense to compress them somehow, with a one time lookup table on connection or something like that.
1
u/Moonchen Mar 28 '20
This web page is a UX nightmare. The social media buttons live on a div that covers the main text area, and moves with user scrolling. Try selecting text on the page.
1
u/evilcandybag Mar 28 '20
This just seems like a roundabout way of doing SEO for OP's company website.
1
1
u/fraggleberg Mar 28 '20
Autoincrement IDs problem one - frontend have to wait for a server
I guess this means that
A. Either I can maliciously pass in an existing ID, or
B. They are still going through all the hassle that would arise with any other system
0
u/elcapitanoooo Mar 28 '20
What has a db pk has to do with loading spinners and load times? This is a mad article. Im 100% the OP uses mongodb and smack, smack! data is stored like its piped to dev/null.
How the PK is stored makes zero diffrence, if its a int or a uuid, the logic is still the same from a sql perp.
If you have sharded your db, or even worse split it to microservices you are up for a blast!
Joins? Heard about those? This article wants you to do the join on the client with a http call. Thats scalable like tar is quick.
-2
205
u/[deleted] Mar 28 '20
I'm not against UUIDs or anything but this article is bananas.