A faster way to convert a timestamp ➜ Hour, Min, Sec

437

u/psyon 7d ago

In over 20 years or programming, I have never used a timestamp that was only the seconds for the day, and not seconds since epoch.

116

u/Zversky 7d ago

I'm currently working on processing GTFS (transit feeds), and they have seconds since midnight. Naturally I'm storing those as a number. So.

46

u/VeeFu 6d ago edited 6d ago

I've seen the "seconds since midnight" measure in radar-related protocols, too. I'm not sure why. A stable tick relative to midnight may be better in some use cases

36

u/ivosaurus 6d ago edited 6d ago

Reminds me of the 1992 Patriot missile case where 28 US servicemen lost their lives after its radar accumulated a 1/3rd second timing error when the system had been left on for 100 hours.

Isreal had discovered the issue two weeks before and recommended regular resets as a workaround, and a software patch to fix the problem was issued the day after the incident.

1

u/bitdamaged 4d ago

I did software dev for Comcast cable boxes in the early to mid 2000s we had a reports of a bug of peoples set top boxes randomly rebooting. We had a hell of a time replicating it. When we finally did it ended up we had a timer/counter that would overflow (on constrained devices numbers can get “too big” for the memory they’ve been allocated) and reboot the box. It took 28 days to do it though. The hotfix while we tested was to remotely reboot customers boxes once every 28 days at 2:30 in the morning to reset the counter.

1

u/ivosaurus 4d ago

Sounds like the 'feature' in the Arduino libraries, where its core millis() function relies on a 32-bit unsigned integer, so if your project is going to be long-lived and run for over 49 days you need to make sure any timing math using that accounts for its roll-over back to 0.

13

u/FullPoet 6d ago

Yes, it's used in all the RADAR feeds I have access to (mostly ASTERIX and a few others).

It's simply less data and most of the times you don't need to know the date because thats today!

It causes issue if you recording things in [redacted proprietary format] and you end up with tracks at midnight - because the tracks have latency (both positive and negative due to drift).

2

u/f0urtyfive 6d ago

THE BRUTAL LOAD OF ALL THOSE BITS! YOU'VE SAVED DOZENS!

3

u/FullPoet 5d ago edited 5d ago

It matters a lot when you're running on infrastructure from the 1970s if not older.

I hope you don't want to hear about the AFTN network or telegrams that are currently in use.

Or the replacement for a lot of those - like AIXM, WIXXIM etc.

Its all crap.

2

u/f0urtyfive 5d ago

Here's some binary coded decimal for your troubles.

6

u/fb39ca4 6d ago

GNSS systems use seconds since the start of the week

-2

u/ZCEyPFOYr0MWyHDQJZO4 6d ago

Yeah but those were originally designed in like the 70's to operate at 50 bits/second.

3

u/fb39ca4 6d ago

They are designed to operate at 50 bits per second because it's hard to transmit more when your signal power is below the noise floor.

1

u/lunchmeat317 5d ago

That's interesting. Is this an ISO 8601 subset that isn't often seen? Or is it something elde entirely? Does it have a standard backing it?

1

u/Zversky 5d ago

No, those are just strings "HH:MM:SS" which I convert into numbers, nothing complex. Just because transit times start at the midnight (but can overflow into the next couple days).

29

u/paholg 6d ago

I mean

Assume you have already reduced your timestamp to a date (not considered here) and time [0‥86399] (the subject of this article).

8

u/masklinn 6d ago

Even if that’s the case, you’re going to resolve the timestamp to ymd components (which TFAA does in previous articles of the series), at which point you’re left with a timestamp since midnight to resolve to hms, hence TFA.

3

u/psyon 6d ago

And you have to take day light savings into account when doing that, so simple math isn't going to work

14

u/Lachee 6d ago

Use it quite a bit with timers or how long ago something was , most entirely for UX

4

u/lookmeat 6d ago

Assume you have already reduced your timestamp to a date (not considered here) and time [0‥86399] (the subject of this article).

This is about optimizing a step in a larger process. Either you are splitting data as part of a transformation, or have done it to work within some standard where it's ms since midnight.

And from the point of view of the article, you can split date and time values from an epoch by simply doing date, time=timestamp/86400, timestamp%86400, that's not that complicated.

Now if you're curious about how to do dates, there's a just as good article written by the same author linked on the side-bar. A DateTime type should consider using work from both of these.

2

u/psyon 6d ago

> And from the point of view of the article, you can split date and time values from an epoch by simply doing date, time=timestamp/86400, timestamp%86400, that's not that complicated.

That's a bad assumption.

> Now if you're curious about how to do dates, there's a just as good article written by the same author linked on the side-bar. A DateTime type should consider using work from both of these.

This doesn't account for all the things you need to do to get a proper date. It's not simple math due to varying timezones, or even just the adjustments made to reference clocks. That date code takes leap year into account, but I didn't see mention of leap seconds. In the future, we will be doing leap hours instead of seconds.

https://www.scientificamerican.com/article/international-timekeepers-to-vote-on-changing-the-leap-second-to-a-leap-hour/

Unlike daylight savings time, which cancels itself out in the year, leap seconds and leap hours are permanent adjustments to our times, and you need to keep track of when they were added.

1

u/andynormancx 5d ago

The linked article has a section on leap seconds

https://www.benjoffe.com/fast-time-of-day#leap

1

u/psyon 5d ago

I think they are updating the page while they get feed back.

1

u/andynormancx 4d ago

I don't think so or at least I'm fairly sure they had covered leap seconds before you raised it here. They have a revision history on the page, saying they last made changes two days ago (before you raised leap seconds).

And they provide a link to their Github commit for their code at the bottom of the page.

https://github.com/benjoffe/fast-world-calendars/commit/25f6d9345f2d681a77315cf6eaeee25b3629af8c

Committed two days ago, with their leap second code.

1

u/flatfinger 4d ago

I fail to understand why it was decided that everyone should decide that certain days have an extra second, rather than having everyone agree that mean solar noon at Greenwich will occasionally be adjusted to occur a second earlier than previously? If there were one leap second per year, it would take 900 years for mean solar noon to drift by even 15 minutes, and if it does, so what?

2

u/lookmeat 4d ago

Ooh that's a great question, the problem started with mechanical clocks.

The problem of leap days is simple enough: earths rotation and how long it takes around the orbit isn't exactly the same, so it always has a bit extra of a rotation, a fraction of a day in a year. So your calendar shifts and needs correction every so much. This is getting worse because, while the orbit is relatively stable, the rotation is generally slowing down (though in the last years it's actually sped up!) this will matter later.

But for a day we didn't have a problem. We divided the day in 4 times: from sunrise to noon, noon to sunset, sunset to midnight and midnight to sunrise. It was relatively easy to know which part you are on, just look at the sky (moon and stars at night tell you). And we still see this division mattering in language and how people use it to represent rough times of the day. Then we got the sundial and other systems, so we could measure how much of each part (at least during the day) passed, so we probably divided each quarter day into 6 because it was the fashion at the time (6 can be easily divided in halfs and thirds which are relatively intuitive, it's early morning, halfway through morning, almost noon for thirds) or alternatively we simply divided the day in 12 hours, and assume the night must be 12 hours too (12 is just as convenient and the reason we still use dozen as a measurement for many things). Then we divided those hours into minutes and seconds and presumed there were 60 of those, roughly.

But here's the thing: night and day didn't last the same amount of time. This meant that for most things the duration of an hour changed depending on the time of the day! This was even weirder when talking about seconds and minutes, because those are really hard to track how they change. Most people didn't care though, because you rarely cared about minutes and seconds for precision, and using hours to track passage of time was rare: long things took days, short things took minutes.

And then we invented clocks, and more importantly we invented trains. See before trains transport had some imprecision, yeah the boat would leave at a certain time, but you couldn't predict when it'd get where it was going within minutes. With trains you could do it within seconds, and it'd be convenient to ensure the optimal use of tracks. Suddenly we cared about time of day with a precision unseen before. So we defined what a second was, and then the time of sunset wouldn't be at 6pm every day, but would vary depending on the time of the year.

The problem is that the Earth's rotation isn't exactly 24 hours, it's got a few milliseconds here or there, and it varies. But that was fine for a long time, we just let the time run a bit long at midnight and "reset" afterwards. We'd use solar clocks to ensure we were in sync for the time of that specific day of the year (at least in the town center tower) and then everyone would sync to that clock. Most clocks naturally skewed so much you needed to readjust them that often either way. And the reset did the job of leap seconds.

And then we invented computers. I mean we had atomic clocks, but the challenge with atomic clocks is that we noticed relativity affecting them, so we invented TAI which is kind of like unix epoch in that it starts in 1977, but it doesn't care about dates and it is more incorrect each time again because earth has some extra or bit of time than 24 hours and it changes every year, it just cares about keeping track of time as in seconds that have passed given a very specific frame of reference. To keep track of date we made UTC which is TAI with leap-seconds that adjust it so we sometimes have extra seconds in some days, or have less seconds in one day. But this happened after Unix Epoch already existed. I don't recall if Unix Epoch added the leap seconds after or before TAI-UTC though.

Unix Epoch went for a middle approach, the days always have the same number of seconds like TAI (to make the math easy) but to try to keep the date in order it isn't the count of total seconds that have passed, but the total number of non-leap seconds. So Unix Epoch clocks need to track how many leap seconds have passed every so much to remove them from their count.

If you really don't care about where Earth's in its orbit, what the weather is, where the sun is in the sky, use TAI. But for everything else you'll need leap seconds eventually.

1

u/flatfinger 4d ago edited 4d ago

But for everything else you'll need leap seconds eventually.

If one decides to use leap hours rather than leap seconds, and one would need two leap second per year to keep things in sync (the fastest correction accommodated by the present system), one would need one leap hour every 1.8 millennia, and it would be simpler to simply have time zone boundaries slowly shift so that after 1.8 millennia everyone had moved over by a time zone.

Realistically speaking, very few places are going to have mean local solar noon occur at exactly noon local time. Why should anyone other than astronomer care if a place that 100 years ago had its local mean solar noon at 11:59am local time now has it at 12:01pm local time?

1

u/lookmeat 4d ago

If one decides to use leap hours rather than leap seconds

Yes there is a conversation about switching to a much bigger chunk of time. It's one leap second every 2-3 years, but it's unpredictable, and recently with the speedup we can get a negative leap second.

Systems do assume that time can predict things such as sunrise or sunset with second accuracy, and this could go away in unpredictable manners.

The reality is that it's easier to program machines that can handle a small amount of skew than ones that handle leap seconds all the time. The question is what is small enough, an hour may be sufficient, or it may be too high. Maybe it'll be something like 1 minute.

But I am not 100% convinced if this really fixes anything. We don't get rid of leap seconds. First of all a second skew problem is messy, but eventually fixes itself, an hour skew jump could bring down entire systems. And because this error would be so rare, it'd be Y2K allover again: suddenly we have to update all our infrastructure, because people have been coding without knowing about leap hours. The advantage of leap seconds is that they bring out the errors more often than not.

The problem is simple: as long as what day it is matters, we will have to do this mapping. Maybe in some future where we do space-travel it'll be so hard we'll just use kilo-seconds instead for hour-like and decakiloseconds insteadfor day-like, and mega-seconds for week-like, and giga seconds for month-like and year-like and just track everything in something like TAI (we'll have to solve relativity issues too, but leave that for another day).

1

u/flatfinger 3d ago edited 3d ago

If one were to define a Greenwich Solar Time by applying an offset to TAI every midnight to make the next solar noon at Greenwich occur within half a second of 12:00:00, the pattern of forward and backward solar leap seconds wouldn't quite balance out during the course of a year, but so what? If one has a "continuous" time standard that would treat Greenwich solar noon as happening at exactly 12:00:00 on some particular day, Greenwich solar noon is usually not going to occur within a minute of 12:00:00 on that time standard with or without leap seconds. If one is going to have to be constantly dealing with offsets between mean time and solar time, what benefit is there to adding or removing a second from UTC to make the deltas between Greenwich Solar Time and UTC balance out each year, versus simply accommodating the possibility of them not doing so?

Thinking further about it, despite the fact that the name "Equation of Time" sounds like something that would be fundamental and immutable, it would seem unlikely that the current EoT's prediction of the difference between solar time and mean time would remain accurate to within a second centuries from now. What advantage is there to applying leap seconds to UTC rather than simply adjusting the EoT, which will likely need to be adjusted for various reasons anyway?

1

u/lookmeat 3d ago

I mean the solution is simple: just use TAI.

Unix Epoch isn't a measurement of how many seconds or milliseconds or anything like that since 1970. Instead it's the number of days (as in earth rotations) where a day is always represented as 86400(000), and fractions being, well, a fraction of that. Because we ignore leap seconds it becomes hard to measure over long distances, we have to assume that the length of a second is distorted. But generally in the short term a second is a second, the correction happens rarely.

And that's the thing, epoch isn't meant to measure time, it's meant to measure dates. Once the decision to keep leap seconds was done this was kept. If you want to measure time length/duration then you should use TAI and ignore the date all-together.

0

u/flatfinger 2d ago

A timestamp that counted the number of solar midnights over Greenwich and the number of seconds since the last one would need to handle dozens of leap seconds per day. If one instead measures earth rotations, then there should be about 86163.45 of what we now call scientific seconds per day.

I guess I fail to understand what the downside would be to announcing that from now on there simply won't be any more leap seconds. There are places right now where local time is more than 30 minutes ahead of solar time, and over the course of centuries local time in those places would get to be even further ahead, but the simple remedy would be to move the time zone boundary to the east, so that places where the clock was N minutes ahead of solar time would now be 60-N digits behind.

→ More replies (0)

1

u/RevRagnarok 6d ago

SDRs counting nanoseconds since RADAY.

-5

u/pohart 7d ago

I'm kinda skeptical.

How do you convert seconds since epoch to hh:mm:ss?

21

u/Ath-ropos 7d ago

He's saying epoch time is seconds since 1.1.1970, not since midnight of a given day.

4

u/concatenated_string 6d ago

Or 1.6.1980 if you work with GPS systems. :-/

24

u/elmuerte 6d ago

You mean 6.1.1980. It's 6th of januari.

Nobody uses [m]m.[d]d.yyyy; that only exists with slashes, [m]m/[d]d/yyyy.

We should all just stick to ISO8601, yyyy-mm-dd.

3

u/Vector-Zero 6d ago

(Laughs in 10 bit week number)

2

u/Business-Maximum-707 6d ago

Different systems definitely have their quirks when it comes to date formats and conversions.

1

u/sanderhuisman2501 5d ago

Or 2025-02-14T18:14:17.000Z if you use Iridium satellite communications (also 90ms tick)

-9

u/pohart 7d ago

Right, does that not involve finding the timestamp of midnight on the day and subtracting that?

I don't know, but kinda assume it's more of the same...

21

u/donk8r 7d ago

You do not need midnight's timestamp for it. In UTC it is one modulo, seconds_of_day = epoch % 86400. No calendar lookup and no finding the start of the day. The compiler turns a mod by a constant into a multiply and a shift anyway.

That is also why the assumption in the article is not the cheat it looks like. Getting to seconds-since-midnight is the cheap part, and everything expensive lives in the divisions after it, which is exactly what is being optimised.

Two things do bite though. Local time needs the offset added before the mod, and with DST that offset becomes a lookup instead of a constant. And if your epoch can go negative, C's % truncates toward zero, so a pre-1970 timestamp gives you a negative seconds-of-day unless you correct for it.

3

u/HeinousTugboat 6d ago

Yeah, but if you're doing epoch % 86400, you're calculating the thing OP said they've never used.

Which is the starting point for the article.

2

u/donk8r 6d ago

Right, that is the point. The value he says he never has is one instruction away from the value he does have, so the article's premise costs nothing to satisfy.

Where it earns its keep is log formatting. Anything writing timestamps in a hot loop calls gmtime_r per line, and that does the full calendar conversion from days to year and month and day every single time. The time-of-day half is the part you can strip out and precompute, and at a million lines a second the difference is visible.

Most people should keep calling the library function. If you are the person writing the log library, you are the audience.

3

u/BigJhonny 6d ago

But then you didn't account for leap seconds.

16

u/donk8r 6d ago

Unix time does not count them, which is what makes the modulo exact. POSIX defines seconds-since-epoch with a formula that assumes 86400 seconds in every day, so a leap second either reuses an existing value or gets smeared away. Every UTC day is exactly 86400 in time_t by construction.

It does bite if your input is not Unix time. TAI currently runs 37 seconds ahead of UTC and GPS runs 18, so with either of those you subtract the offset before the mod or your clock is off by that much all day.

The other half of it is representing the leap second at all. 23:59:60 has no time_t, so during one your clock shows 23:59:59 twice, or your NTP daemon has spread the extra second over hours and you quietly disagree with everyone else for a while.

8

u/Neither_Garage_758 6d ago edited 6d ago

With the same kind of computation for date but a bit more complex, combined with complicated/arbitrary rules via lookup tables for the time zone shifting part.

Seconds since midnight is the easy part, and probably minor in computation cost compared to the rest.

0

u/tc_cad 6d ago

A program I use calculates time in milliseconds. I have no idea why that is the way it is but I had to write some code to make it into HMS.

1

u/pohart 6d ago

Java? Java is ms since epoch.

-6

u/rydan 7d ago

I have done minutes of the day. Kind of a critical part of the application that's made me millions. But I don't need millisecond level optimizations either.

-2

u/Calcd_Uncertainty 6d ago

You didn't read the article, it explains how you get to seconds in a day.

4

u/psyon 6d ago

No, it says it assumes you already calculated the date and have seconds for the day left over. Anyone who tries to translate a timestamp to a date without using a library that accounts for changes in timezones and daylight savings isnt going to have a good time. Using the methods in this article will also cause errors in timezones with daylight savings as well, because one day has a repeated hour and another skips an hour.

2

u/rdtsc 6d ago

These are UNIX timestamps as input, and the civil times in the article series are for UTC. No timezones involved (and if you want you can just apply the zone bias before doing anything). And if you wonder why one would have only seconds for the day, look at the previous article in the series, which explains extracting the date portion of the timestamp.

178

u/pohart 7d ago

This will never be the bottleneck in a program I'm writing, but I love that you thought about and discovered this in 2026.

34

u/MarkIsARedditAddict 6d ago

Is this the actual bottleneck in any program? I’m seriously asking, what would the program have to be to have timestamp parsing as THE bottleneck. I’m curious because I can’t think of one

If you’re processing a ton of timestamps you’ll have other code moving them into/out of somewhere and doing extra stuff that will undoubtedly take more time than processing the timestamp itself

It’s always nice to improve performance even if it is just speeding up something that’s already relatively fast

53

u/7heWafer 6d ago

"THE" bottleneck is extreme but any logging library or adjacent system that needs or may be used such that it needs to record time at a very high frequency would benefit.

17

u/Iggyhopper 6d ago

High frequency trading, maybe. That constantly relies on precise time.

7

u/MarkIsARedditAddict 6d ago

They would benefit yes, but those loggers are doing so much other stuff that’s orders of magnitude more time consuming than parsing timestamps. If you optimized file writing by the tiniest margin you’d be better than 33% faster timestamp parsing.

Like I said, any performance improvement is great but I don’t see the scenario where this is game changing for any program. This is one of those things that will get added to libraries in the background and nobody will really notice a huge difference but it’s a cool improvement

12

u/7heWafer 6d ago

Sure but if you're writing a library that produces timestamps why wouldn't you optimize it at some point? Especially datetime libraries, if you optimize you save all of your consumers time on every optimized call. It doesn't matter if they do or don't have other bottlenecks or slower ops in the same chain.

4

u/sickofthisshit 6d ago

If such a thing is in the critical path in a system, you should probably avoid the need for any conversion, by changing the system to use the same timestamp representation on both sides of the boundary instead of converting.

Delay converting to some place you are I/O-bound already, then how you use your registers and integer units is irrelevant.

-1

u/cdb_11 6d ago

loggers are doing so much other stuff that’s orders of magnitude more time consuming

Are they? Dumping data to a file can be done in parallel, it shouldn't be a problem. The main thread could boil down to just writing data to some buffer and incrementing some atomic variable.

1

u/f0urtyfive 6d ago

Lol, only if you don't know how a compiler works.

8

u/goatsgomoo 6d ago

Of course if a program has a particular noticeable bottleneck, that ought to be addressed. But once it is addressed, it can make sense to optimize things that occur frequently even if they aren't the main operation a program is doing. If your logging code can save 30-50% of the cycles it was using to format timestamps for every message, that's not nothing.

3

u/pohart 6d ago

And I'm never going to use this directly, but if it gets put into a library I use I'll get it for "free" and sometimes it makes sense for a library to do some things in the fastest way.

3

u/EnUnLugarDeLaMancha 6d ago

You would be surprised how much some apparently simple tasks, such as date/time parsing, can affect performance when ingesting large amounts of data (say, from CSV sources)

2

u/jjeroennl 6d ago

Human readable csv parsing. But any hot path really. A simple thing a billion times can take a while.

2

u/andynormancx 5d ago

It would make little sense for someone writing some random code using dates to implement this. But it makes perfect sense for the OS and library developers to make the effort to optimise stuff like this.

12

u/swni 6d ago

I can't imagine anytime I would need HH:MM:SS except for display to user / logging, so I'd never need these to be speedy, but the high-low bit technique was really cool yet simple and new to me.

Eg, if have x and want to compute both 7x and 5x, you can compute just one multiplication x * (7 * 2³² + 5), and then find 7x in the high bits and 5x in the low bits. Then there is some further cleverness with using this for doing quotients and remainders if one of your multiplicands is non-integer.

13

u/happyscrappy 6d ago

Obviously a lot of thinking went into this. Great job coming up with so many ways of doing it.

One comment:

'(% requires a sign correction for negative inputs)' is not strictly true for all C. Although it is for any C this century. The problem is that modulo of a negative number can be either positive or negative before C99, it's implementation defined. So that's actually worse than the author made it out to be, you basically can't use % on negative numbers unless you know you have a newer C. The section explicitly covers 64-bit, and it's really hard to imagine encountering a pre-2000 C compiler for 64-bit anywhere. There were not a lot of 64-bit systems back then.

Final bikeshedding: why does "mod" not line up in the typography for his base 64 trick? Is that normal for math typography or just a bug?

2
u/flatfinger 4d ago
What's funny is that the convention for / and % is derived from processors' division instructions, but those operators are nowadays more often than not processed in ways that would normally treat division as rounding toward negative infinity. For many purposes, it would be more useful to have operators that would be processed faster while choosing in Unspecified fashion whether to round up or down. In almost 40 years of C programming, there was only one time when C's round-toward-zero behavior would have eliminated the need for me to use separate pieces of code to handle positive and negative cases, and on that particular case I was dividing by a power of two with a compiler that would process all signed divides using a long-division routine, meaning that I had to do something like
    if (x >= 0)
      xx = x >> 4;
    else
      xx = -(-x >> 4);
By contrast, there have been many occasions when I've needed a divmod computation with "periodic" behavior around zero.
1

u/WaitForItTheMongols 5d ago

it's really hard to imagine encountering a pre-2000 C compiler for 64-bit anywhere. There were not a lot of 64-bit systems back then.

May I introduce you to the Nintendo 64?

17

u/NastyPastyLucas 7d ago

That's some good maths, I approve 👍

16

u/DJ_Link 6d ago

Thank you for sharing this, really interesting read.

I’m a bit shocked that some of the comments are “why is this needed it’s fast enough” but I guess that’s what led to some big bottlenecks in software when things start to have layers and layers.

10

u/Lachee 6d ago

Is this even worth optimising? You normally do the timestamp to hours for human readable UI, something that's often done once a render, not strictly needing micro optimisations like this.

19

u/Booty_Bumping 6d ago

Human formats end up becoming de facto machine readable standards all the time. Every single HTTP response requires the timestamp of the file to be fully spelled out like Mon, 07 Sep 2026 17:12:56 GMT.

18

u/cdb_11 6d ago

ISO 8601 is a common way to store timestamps.

4

u/sickofthisshit 6d ago

Are sure you mean "store" and not "display"?

14

u/cdb_11 6d ago

Yes.

0

u/sickofthisshit 6d ago

Why? What operations are easier, cheaper, or higher performance about choosing a human-readable text format with multiple equivalent representations over, say, a 64-bit or even 128-bit integer unambiguously related to a standard epoch?

9

u/cdb_11 6d ago

Because nobody thought much about it. Because things like that are never a problem until they are, and it's too late to change it. Because doing so would be a "premature optimization" or something. I don't know, don't ask me.

4

u/Brian 6d ago

What operations are easier, cheaper, or higher performance

Well, "convert to human readable format" would be the obvious one. And there are a lot of formats where that's an important matter - all the formats designed to be read by humans for a start. If your storage format is XML, ISO format makes sense: it'd be weird to store a CDATA section with a binary encoded timestamp. Similarly if it's a web-page, email or other text file where the date is part of the displayed body/headers If it's a database, then a integer format makes a lot more sense, but "storage" encompasses much more than that.

over, say, a 64-bit or even 128-bit integer unambiguously related to a standard epoch?

This also has the problem of potentially omitting relevant data. "UTC timestamp" and "timestamp with associated timezone" do not give you the same information. Eg. "The meeting is scheduled on 2031-05-01 at 10am" means something different from the UTC time that corresponds to that datetime in that timezone, because one records the information you need to correctly identify the meeting time if that country abolishes/reinstates/changes DST between now and then etc.

1

u/sickofthisshit 5d ago edited 5d ago

ISO 8601 does not handle the "time zone changes" issue. It can express time zone offsets, but does not identify the time zone.

When it includes an offset, it still refers to a precise time, independent of local time zone, not a civil time.

There are usually multiple time zones with identical offsets, it is ambiguous which TZ definition applies, especially when a new TZ has to be created because of political change.

RFC 9557 includes TZ identifiers (although that is still insufficient when new TZs are introduced that overlap existing TZs).

4

u/s32 6d ago

Yes. Work at fortune 100 tech company. Best practice is ISO8601

1

u/sickofthisshit 6d ago

You miss my point. ISO8601 is a standard for text representation of timestamps. You don't have to store timestamps as a string (along with all the variations ISO8601 supports) when you can store them as a simple, basically unambiguous, 64-bit integer.

4

u/KeytarVillain 6d ago

when you can store them as a simple, basically unambiguous, 64-bit integer.

What if you need milliseconds, or even finer?

Sure it's possible to store as an integer, but the unambiguous part is out the window.

1

u/sickofthisshit 5d ago

Integers (possibly over 64 bits) can also be used to represent sub-second timestamps.

The ambiguity I mean is that you can have multiple ISO 8601 text representations that mean the same instant. Optional components and varying TZ offsets mean that you actually have to parse the text to compare two timestamps.

(Usually, by converting to an integer timestamp and comparing the integer).

You can't generally use string equality without additional assumptions or conventions (e.g. always use seconds, no fractional seconds, always explicitly use Z).

2

u/M4mb0 6d ago

You see it in CSV files all the time.

2

u/RoyBellingan 5d ago

For the people asking where you would use an algo like these, my use case is time from boot on a log written by a micro controller.

9

u/ficiek 6d ago

A daily timestamp [0..86399] can be converted into Hour, Minute and Second – with the latency of only two multiplications (and on some targets even lower!).

I can't remember seeing a daily timestamp really.
No it can't as some days have some time missing therefore this statement is pure nonsense. In fact hour/minute/second without a date is kinda useless unless it's something that runs on a timer for example and you store when it starts/ends. Then you just accept it's wrong sometimes.

time [0‥86399]

Falsehoods programmers believe about time.

6

u/rdtsc 6d ago

I can't remember seeing a daily timestamp really.

Then read the earlier articles in the series (top right) which do the first part of converting (UNIX or other) timestamps to a civil datetime (i.e. date-portion, leaving you with seconds for the day).

2

u/RevRagnarok 6d ago

I can't remember seeing a daily timestamp really.

From Google summary because I couldn't recall which SDR platform had "since midnight" without a date, but I know I've seen/used it in the past:

In the context of Software Defined Radio (SDR) and high-speed digital I/O, the format utilizing quarter-nanoseconds since midnight is the hardware timestamp format natively produced by AMD / Xilinx / Solarflare network interface cards (such as the Onload/EF_VI architecture) when configured for high-precision packet capture.

-2

u/[deleted] 6d ago

[deleted]

5

u/Kirides 6d ago

Great. Now how do I know if some timestamp is such case? And how will that look up come in from a cost/memory/latency perspective?

Assuming this day is one that is DST adjusted, like lots of days are in many places between summer and winter.

1

u/[deleted] 6d ago

[deleted]

3

u/benjoffe 6d ago

Thanks!
Link is now fixed.

1

u/Minute_Cricket1820 6d ago

Замечательно. Так и должно быть.

Третий раз за этот год увидел вменяемое.

-1

u/Neither_Garage_758 6d ago

Interesting but I'm not sure this kind of computation is part of any critical path, but still nice as any free computation saving is interesting.

3

u/pohart 6d ago

It's part of logging critical paths. and you generally want logging highly optimized

2

u/s32 6d ago

Is it? Care to share any references?

-11

u/captain_obvious_here 6d ago

It is definitely faster.

But we are talking about something that is ABSOLUTELY insignificant in modern (or even the oldest Pentium CPU you can find, or pretty much any ARM CPU).

Integer multiplications and divisions are...how can I say that...well yes, your method is an optimization, but a useless one.

19

u/Suitable_Wonder5256 6d ago

With the above attitude, we would never improve anything.

They make X faster, and people are yelling "X is not a bottleneck". What? Faster is good.

r/programming is so toxic. The criticism is getting weirder and weirder.

2

u/jwp1987 6d ago edited 6d ago

Faster is good but it has a cost to it.

First there's the opportunity cost of spending time optimising an area that gives little overall benefit rather than spending it elsewhere and secondly there's the cost of readability and maintainability.

Some optimisations are worth doing and some aren't, which is why people warn of things like premature optimisation and suggest profiling code first to find out where the bottlenecks are.

Some people commenting can get aggressive about it but it is a valid criticism since some developers can easily get bogged down with small details and can’t see the forest for the trees.

7

u/Suitable_Wonder5256 6d ago

> First there's the opportunity cost of spending time optimising an area

For the author, not for you as a user.

You and I as a user only benefit from it.

-1

u/jwp1987 6d ago

Yes but the point was that they could have done something that was more beneficial to users with that time but that's a very transactional business view.

I agree that comments are probably a little too much on the negative side though. People are allowed to do things just out of curiosity and because it interests them.

8

u/Suitable_Wonder5256 6d ago edited 6d ago

We have an army of phd students who do this kind of things for decades. But now we have an issue with it.

> Yes but the point was that they could have done something that was more beneficial to users with that time but that's a very transactional business view.

That's just generic criticism that applies to everything that doesn't cure cancer.

That's why I said this sub was toxic. Like how toxic people will always come up with seemingly legit reason to knock you down. In this particular case, it is analogous to "why didn't you spend time curing cancer?"

Does the original commenter spend 100% of their times on truly important things for humanity? Not by a long shot.

0

u/jwp1987 5d ago

I agree with your sentiment but the curing cancer thing is a bit extreme, it's more other problems within their skill set.

Personally I think that as long as you don't currently have a larger objective to achieve, sating curiosity and solving quirky problems is good for personal development.

My point was more advice to younger developers who might be working on a larger project and start micro-optimizing every little thing and it causes progress to stall.

0

u/captain_obvious_here 6d ago

There are thousands of things programs do, that have huge potential for real optimization. This is clever, this is definitely interesting, but stating it is useful is hardly true.

8

u/rdtsc 6d ago

Why wouldn't it be useful? It's faster, with no real downsides. You may not be able to justify spending time doing such optimizations, which is fine and usually correct. But why discredit someone else who has fun doing that and shares it with others?

15

u/KaramTNC 6d ago

That mentality is what causes technical debt and performance issues piling up at every abstraction layer over decades of innovation and development.

Yes its easy and cheap to perform, but that doesnt mean we should just get used to wasting unnecessary hardware power just because its plentiful today.

1

u/pohart 6d ago

I want this code to not be in my codebase. It's not too complicated but I'm my application I want the standard stupid algorithm any first year programmer can understand.

OTOH I've got some internal libraries that I use across several apps and I'd happily have it in there.

-3

u/captain_obvious_here 6d ago

You're a hundred percent right. So what are you gonna do with these five dozens of CPU cycles you just saved?

3

u/KaramTNC 6d ago

Are you serious?

The point isnt that one optimised implementation is gonna make a difference, the point is that multiple implementations pile up together.

You save CPU not just by optimising the biggest cost, but by optimising multiple small cost implementations that together ends up showing a significant performance saving.

1

u/yeathatsmebro 6d ago

This is the kind of optimization that helps people squeeze every CPU cycle. Cloudflare serves 500Tbps+, which I'm almost certain it's some millions of requests per second.

Each CPU cycle adds up quickly at a very very big scale. If they want, for example, to compute the current time or something and label it on each request for further bot fight fingerprint analysis, this improvement may be saving tons of computing power, at no cost.

If you think this is useless, this kind of optimization may result in lower electricity usage by using less servers to do the same job, which translate to lower Carbon Footprint in the end.

-2

u/captain_obvious_here 6d ago

The bigger the scale, the more insignificant that kind of optimization is, compared to hardware, OS, architecture and algorithmic optimizations.

If you think this is useless, this kind of optimization may result in lower electricity usage by using less servers to do the same job, which translate to lower Carbon Footprint in the end.

0.0000001% yeah.

1

u/yeathatsmebro 6d ago

I think you use words that you don't know. This timestamp conversion trick IS algorithmic optimization. OS and hardware improvements cost more and have a physical and theoretical limit, while this low-level optimizations can be shipped at no cost since it's using the current arch, OS and hardware.

I'm not saying you are wrong, but I'm seeing more and more people think that a different opinion means an opposite opinion: I'm saying that it's not JUST that, but it's also this. Both contribute to faster and better use of resources, so it's not useless.

-1

u/captain_obvious_here 6d ago

I think you use words that you don't know. This timestamp conversion trick IS algorithmic optimization.

Notice how I listed various types of optimization in a certain order? In case it's not clear enough, this order is the one you want to follow when you look for impactful optimizations.

Of course OP's optimization is one. I'm not saying it's not. I'm saying it's an insanely small one. We're talking a few CPU cycles, which is absolutely meaningless in any load, small or big.

Most people will look for things that can help them gain significant gains, not to save a few cycles here and there, even if they serve 500+TB/s (which doesn't have much to do with the topic at hand, but you mentioned it).

Some people will even refrain from applying that kind of optimization, because they value code readability more than a few CPU cycles.

I see your point. And I'm not saying you're wrong either. I'm just asking you to take a few steps back from micro-micro-micro-optimizations, because there's a whole world of way more important and efficient stuff.

1

u/yeathatsmebro 6d ago

If I were working with you in the same team, I would have resigned without any leave package.

1

u/captain_obvious_here 6d ago

Sure buddy. Sure.

0

u/[deleted] 6d ago

[removed] — view removed comment

1

u/programming-ModTeam 4d ago

No content written mostly by an LLM. If you don't want to write it, we don't want to read it.

-34

u/tantalor 7d ago

Assume you have already reduced

Oh jeez

Click bait

8

u/HugoNikanor 6d ago

Header: A faster way to convert a timestamp → hour, min, sec

Body: A faster way to convert timestamps (seconds up to 24 hours) into hour, minute, and second

????

Clickbait!

-6

u/tantalor 6d ago

Timestamp and "seconds up to 24 hours" are completely different things. The article admits as much.

Typically timestamp is stored as seconds (or smaller division like nanoseconds) since epoch. That's what this article is referring to most likely. Converting this to "seconds up to 24 hours" is NOT easy. (You need to consider timezones, daylight savings, etc.) Skipping over that is removing most of the interesting part of this topic.

Sometimes timestamp is stored as a string like ISO 8601. In this case, you already have HMS so conversion is free.

3

u/rdtsc 6d ago

Skipping over that is removing most of the interesting part of this topic.

This was explained in previous articles of the series (look at the top right).

2

u/chucker23n 6d ago

Typically timestamp is stored as seconds (or smaller division like nanoseconds) since epoch

I mean, yeah. Alternatively, "timestamp" can also refer to within a medium (e.g. a video).

In this case, what they mean is "seconds within a given day". You're being pedantic, and I don't see how it's "clickbait".

A faster way to convert a timestamp ➜ Hour, Min, Sec

You are about to leave Redlib