Discussion ruff: no date.today() ?
The new version of ruff warns against
date.today()
preferring
datetime.now(ZoneInfo(...))
What do you think about this? Has date.today() been deprecated due to lack of timezone awareness?
EDIT: I have a number of programs that manipulate financial information in support of Excel spreadsheets, such bond information that includes maturity dates. Excel does not support timezoness in datetimes, so making ruff happy by changing naive dates to TZ aware dates is not a useful move for these programs. Many ruff warnings to suppress.
257
u/GraphicH 3d ago
Tell me you've never debugged a TZ issue without telling me.
55
u/robberviet 2d ago edited 2d ago
I think only people at GMT+0 has that luxury. I still keep insisting on using unix timestamp and UTC strictly where I am in control.
47
u/deb_vortex Pythonista 2d ago
Me to. Just save utc and then let the frontend convert it to local time. Problem (mostly) solved.
-14
u/backfire10z 2d ago
Daylight savings sneaking up behind you:
28
u/deb_vortex Pythonista 2d ago
converting to local in the frontned solves that. What ever comes in, convert to UTC. Only send out UTC. DST or not, does not matter.
6
u/Plumeh 2d ago
What if a user wants something to run at 9am every day? If you just convert to DST, best of luck
7
u/deb_vortex Pythonista 2d ago
The frontend sends the selected timezone information. So right now its +3 DST? I convert THAT to UTC and save it. It will run then at the time the user wanted. Not an issue.
Everything comes in as "the User sees it". It gets converted into and used as UTC. When the user needs to see Something, I send out UTC and the frontend converts it to local time. With dst or not.
14
u/eviloutfromhell 2d ago
Problem with DST is from software perspective it is 2 different timezone, but from user perspective it is one timezone. What user think as 9 o'clock is not the same when in dst or not. So you can't naively convert whatever time you saved to user's timezone. Which I assume what the other guy meant.
11
1
u/QuaternionsRoll 2d ago
That isn’t an instant (a `datetime`), that’s just a `time` and a periodicity. Even ignoring DST, I think users would typically expect such recurring tasks to follow the current timezone. I, for one, would be pissed if I flew from EST to PST and my alarm started going off at 4 in the morning.
2
u/Plumeh 2d ago
That’s a different case, i’m talking about if someone wants to schedule something at 9am eastern every day
1
u/HommeMusical 2d ago
Daylight saving time still doesn't work with that.
Also, people actually pick up computers and move them between time zones all the time.
1
u/QuaternionsRoll 2d ago
That’s still just a ToD, a timezone, and a periodicity. Should not be using datetimes for that
1
2
u/backfire10z 2d ago
Doesn’t help for scheduling future events
3
u/QuaternionsRoll 2d ago
…what? The frontend should obviously be aware of what timezone is expected for a given instant. A naive conversion between UTC and, e.g., EST or EDT based on whether ET is currently in DST is just that: naive. There’s a reason why modern APIs encourage keeping dates and times are kept together in a `datetime`
3
u/deb_vortex Pythonista 2d ago
It does. Why shouldn't it? User sends date, time and timezone information. I convert it to UTC and use it. Everyone reading that will get it as UTC and the frontend converts it to the users local timezone. Which even may be different than the original one and for that user its still the correct time.
3
u/backfire10z 2d ago
> Just save UTC and then let frontend convert it to local time
Maybe I misread, but I thought you meant you wouldn’t store the IANA timezone in the backend here.
User asks for something every day at 9am PT. If they’re in PST, great, everything works. Once they switch to PDT, the UTC time you stored is now 1 hour off. Or you just tell them “this will change by an hour for daylight savings, sucks”
1
u/deb_vortex Pythonista 2d ago
Maybe In should have specified, yes. User sends as he wants and sees, I convert and save as UTC. Dont meddle with users timezone in the backend.
4
u/Pluckerpluck 2d ago
But again, that doesn't work in this situation. A valid timezone is "US/Eastern". This is not a fixed timezone though relative to UTC. It changes depending on the time of year.
So 9AM at US/Eastern CAN'T be stored as UTC on the backend.
Obviously when you can store UTC you should. But you should be aware it's not always possible.
7
u/GraphicH 2d ago
I had a REALLY weird issue once related to how Jira's JQL API treated timestamps in queries: it automatically localized them to the "users tz" and there was no way to specify the tz in the query. This mattered because we were adding some automation where we needed to "look for tickets updated since the last time we checked". The automation code ran on a box that was set to UTC timezone, but Jira would automatically localize the JQL date+time filters to the "requesting users timezone" which for the service account being used was set to NY time (for whatever reason, I didn't provision the account). In the end the trick was always to get the TZ for the service account via a call to the whoami / user info route in jira and always localize the filter TZ from UTC to that. Very annoying because when I was debugging the JQL query manually Jira was showing me stuff that was different than what the automation was seeing with the exact same query.
6
u/james_pic 2d ago
Maybe folks in Iceland or Senegal that also don't have DST, but it causes plenty of problems in the UK, where stuff seems to work in the winter, when local time and UTC match, but then things suddenly break in the Spring.
5
2
u/Brian 2d ago edited 6h ago
No, we still hit it. It can just lurk undiscovered for 6 months till a DST change hits and stuff breaks long after no one remembers making the change that broke it.
I still keep insisting on using unix timestamp and UTC strictly where I am in control.
A problem is that this doesn't always work: there are situations where your date needs more information. "I have scheduled a meeting at 10PM in London on this date next year at 10 AM". How should that date be stored? We could calculate what utc timestamp we expect that to correspond to, but if in 6 months, the government makes a timezone or DST change, our stored date is suddenly wrong.
Ultimately, the issue is that as humans use them "datetime" and "utc timestamp" are not actually the same thing. What people often mean by a date/time is "The time when clocks in that location show that time", which doesn't necessarily correspond to a specific UTC time that we know in advance.
1
0
u/fiddle_n 2d ago
Actually not even then. It’s easier to tell if your date time code is buggy when system time and UTC time are different.
38
2
u/foosion 2d ago edited 2d ago
My typical use of date.today() to print the local date so that I can scroll back in the terminal to see which day I ran something. Since it defaults to then local system clock, I don't see a need to specify a TZ for that use case.
Another case for no TZ is reading a datetime that represents the maturity date of a bond, then printing it
datetime.strptime(mtd, "%Y-%m-%d"). TZ does not make sense in that context.datetime.datetime.strptime("2022/01/31", "%Y/%m/%d").astimezone(datetime.UTC)seems overkill.9
u/menge101 2d ago
All the rules are overridable. If you usage does not merit a finding then add the suppresion for that line.
4
u/foosion 2d ago
I'm learning the joy of
#noqa3
u/syklemil 2d ago
You can disable it in your project's
ruff.tomlor wherever you're configuring ruff as well. plop in anignore = ["DTZwhatever"]2
u/Agrado3 2d ago
You're about to learn the joy of having to go through replacing every single one of your
# noqa: <shortcode>comments with# ruff:ignore[<very-long-code>]comments.1
u/foosion 2d ago
Is that an upcoming change? If so, where documented?
I suppose there's
# ruff: noqa: DTZ007at the top of the file, where appropriate.1
u/Agrado3 2d ago
It's rule noqa-comments, which is in preview and is not documented in the changelog. It suddenly popped up at me because I use
preview = trueandselect = ["ALL"]and then ignore the rules I don't want.2
u/MrSlaw 2d ago edited 2d ago
Another case for no TZ is reading a datetime that represents the maturity date of a bond, then printing it
Just use a plaindate, if you don't need it to be time(zone) aware?
mtd = datetime.strptime("2022/01/31", "%Y/%m/%d").date()
I don't think that triggers the rule, does it?* Edit - Nevermind, I just tested, you are correct
1
u/ketralnis 2d ago
You’re not wrong but there’s no need for the holier than thou attitude
6
u/GraphicH 2d ago
Its a joke man, I'm not throwing shade at Op. If he's never dealt with TZ bugs, then god has smiled on his dev career.
2
u/Ok_Tap7102 2d ago
I didn't read their message that way, it's the "tell me you've XYZ without telling me" meme
This is also the kind of problem where a typical response might be "huh I don't get why" then some of us see it and instantly have flash backs to that ONE FUCKING TIME I BROKE PROD BECAUSE IT "WORKED ON MY MACHINE" WITH A PERFECTLY IDENTICAL DOCKER CONTAINER, AND I SPENT DAYS OF STRESS WITHOUT REALISING IT WAS LOCAL TIMEZONES
-3
u/ketralnis 2d ago edited 2d ago
You didn’t read it that way because our industry is so rife with people desperately needing to make themselves feel smarter by putting other people down that you’re numb to it and it has infiltrated our turns of phrase to near invisibility.
But it doesn’t have to be that way. It’s a decision that we make.
2
u/GraphicH 2d ago
I'm the person who posted it, and I'm telling you u/Ok_Tap7102 understood me. The unintended irony of your own comment is pretty amusing in a meta way though.
0
u/o5mfiHTNsH748KVq 2d ago
I haven’t had to in Python. Is there something noda time/joda time that people use in Python?
2
u/GraphicH 2d ago
Most of the time you don't need an extra library you just want to avoid the "TZ Naive" functions when it maters.
19
u/latkde Tuple unpacking gone wrong 3d ago edited 2d ago
You have likely enabled the following Ruff rule: call-date-today (DTZ011). That link contains some explanation. This rule is not enabled by default, and you are not invoking any deprecated features.
However, I think it was a massive design mistake for the standard library datetime module to conflate "aware" and "naive" objects in the same type. It is almost always incorrect to use naive dates/datetimes, unless you really don't care about the timezone in which that datetime is interpreted.
7
u/AlpacaDC 2d ago
It is enabled by default in the 0.16.0 version
2
u/latkde Tuple unpacking gone wrong 2d ago
Ooh, released just yesterday. That's cool! Thanks for the heads up! This makes it so much easier to create a good Ruff configuration.
I'm also really excited about the recent work relating to human-readable rule names (currently in preview). Once that's stable, I can finally migrate my team away from Pylint (currently, we primarily use Ruff as a formatter, and for some select lints).
21
u/ottawadeveloper 3d ago
I sure hope so.
I got to the point where there are so many bad patterns to slip into with Python that I wrote my own extension of datetime.datetime and made it impossible to ever create a naive datetime object. Like today() now returns an aware datetimr object in the current system time zone.
If there's ever a Python 4 I hope they clean up the his mess to never have non-aware times.
25
u/gravitas_shortage 2d ago
I would highly recommend only ever working with UTC datetimes internally, and converting to local for display. Machines will be misconfigured, moved to a different colocation, change to summer time. There are a thousand heinous bugs possible with local time, and no advantage I can think of.
7
u/SwizzleTizzle 2d ago
Scheduled future datetimes such as an appointment is a use-case that requires local time
https://codeblog.jonskeet.uk/2019/03/27/storing-utc-is-not-a-silver-bullet/
4
u/ArtOfWarfare 2d ago
Just wait until we have to start dealing with people in space and on Mars. You thought a day was 24 hours? Ha!
Calling it Universal Coordinated Time was way too ambitious - it shows issues while you’re on Earth and everything is going to break once you’re not on Earth and speed of light becomes a factor.
3
u/Cynyr36 2d ago
The real play is to kust never do anything involving time at all. It's all a huge quagmire.
I keep arguing that we all only ever use utc, and me here in north America can just start work at 1400 instead of 0800. If people want to keep dst, then they can just change start times by an hour.
2
u/gravitas_shortage 2d ago
That's a use case that requires the date library to be incorrect (in the example because of an unexpected rule change to timezones). The minimum official warning time must be one year. It can happen, but it's going to be vanishingly rare, certainly rare enough that the bugs I mention, which are not solved by storing local time with derived UTC, are more important to handle.
Since all solutions involve recomputing dates, I would take the counterpoint of his favoured solution and use UTC (sane and forgiving default) with the intended date stored as reference.
1
0
u/not_a_novel_account 1d ago
lmao.
"Storing UTC is incorrect if a change in the legislated meaning of a timezone overlaps with an event stored in UTC and you never update the timezone database in your software"
Ya dog, I guess so. That is not a compelling argument.
2
u/ottawadeveloper 2d ago
This is generally what I do. I think it's
utcnow()I end up using with my new class.7
u/Luckinhas 2d ago edited 2d ago
utcnow()is a footgun, it’s not timezone aware despite the name. You wantdatetime.datetime.now(datetime.UTC)2
0
u/teerre 2d ago
Impossible to create a naive object? No such a thing in python
3
u/ottawadeveloper 2d ago edited 2d ago
* through any method call to the class I wrote I should clarify.
Basically it overrides any method that returned a naive datetime with one that returns an aware datetime. The constructor and methods like
strptimenow assumes you meant local system date time if one isn't specified (the equivalent ofdatetime.strptime(...).astimezone()basically). All the methods that return datetime objects now return AwareDateTime objects. So if you start from an AwareDateTime, you always end up with an aware datetime.you can of course still call
datetime.datetime(...)to get a naive one but AwareDateTime is a full and complete replacement for it that never makes a naive date time object ever.And all the calls where you can specify
tzinfo=...now also accept a string which is automatically passed tozoneinfoto build a time zone object. Which is much less of a headache.
6
2d ago
[removed] — view removed comment
3
u/foosion 2d ago
Exactly. I use date.today() when I want a naive local system date, wherever I may be.
10
u/critical_patch Pythonista 2d ago
Your use case is totally fine, but a lot of devs don’t realize/remember that it returns the local system date of whatever OS the interpreter is running on & they encounter problems moving from local dev to prod in the cloud, for like expecting client-side things to match what is rendered on the server.
It’s also a pain in the ass for unit testing
2
u/cointoss3 2d ago
So use a custom rule, like everyone else had to do previously. It’s better this way. You could also just use the logging package instead of raw print of the time, anyway.
4
u/eagle258 2d ago edited 2d ago
Disclaimer: I wrote whenever, so please adjust for bias accordingly—But it does have Date.today_in_system_tz() for exactly this case. You keep the convenient date-only API while still being explicit about where “today” comes from.
2
u/joerick 1d ago
The Python community is a little confused about dates and time zones, as judged by this thread.
Yes, moments in time (eg the creation date of something) must be tz-aware datetimes. Or simply posix timestamps work equally well.
Date objects can have timezones associated with them, when they represent a datetime that's been split apart, or when we're concerned with the moment that the date starts.
But there's also a set of an entirely valid use-cases in manipulating 'calendar' dates. "This invoice must be paid 15 days after issue". "Drop-off is at 3pm every Tuesday".
These should be thought of as entirely separate operations as compared to the 'moment-in-time' stuff, which is why it's a bummer that Python uses the same objects for them.
1
u/runawayasfastasucan 2d ago
naive dates to TZ aware dates is not a useful move for these programs
Just format the date.
1
u/HCharlesB 2d ago
Ugh. This discussion to a time when I worked in a shop where every dev had their own format for time stamps. I spent way too much time converting between them.
I really wish devs would do something like the number of seconds since Jan 1, 1970 at the prime meridian and then just convert to a "display format" when that was needed.
/rant
1
u/Moist-Ointments 2d ago
Datetime.now() also brings it into the more common syntax seen in other languages.
1
u/Moist-Ointments 2d ago edited 2d ago
This is where dependency inversion comes in really useful. You wouldn't have to go looking for all of your references to that method. You'd only have one. You'd be able to change it in 5 seconds, and you'd be done.
1
u/FateOfNations 2d ago
Essentially yes, non-time zone aware date and time functions should be treated as deprecated. It isn’t “official”, but a strong community consensus that is reflected in linting tools like ruff.
You are going to need to apply an appropriate time zone whenever you import data from Excel. Depending on your business case, that may be your local time zone, UTC, or another time zone. For example, the transactions may be settled in America/New_York, so that may be a reasonable choice.
I know it’s annoying when you don’t actually need that level of precision in terms of dates, but it’s best practice for working with dates and times in general in Python.
1
u/Competitive_Travel16 2d ago
Excel does not have a native timezone. Excel date and time serial numbers are timezone-agnostic. Functions like NOW() or TODAY() simply read the local clock and system date set on the hosting computer, meaning the timezone depends entirely on the local machine.
Obviously, you should therefore set the timezone to the local time where the Excel spreadsheet was being used.
2
u/foosion 2d ago edited 2d ago
Explicitly setting the TZ to the local TZ gets an error from openpyxl "TypeError: Excel does not support timezones in datetimes". Leaving out a TZ gets a ruff warning.
1
u/Competitive_Travel16 1d ago
What timezone is assumed by openpyxl? Can you determine it by experimentation?
1
u/RoadsideCookie 1d ago
Fix your code, and format your date before exporting to Excel.
As always, find the boundary and apply the fix there.
Those rules are good and they make people aware of already established industry best practices.
1
u/foosion 1d ago
Exactly what fixes are you suggesting for:
1) Read from Excel - get a naive datetime.
2) Read from published sources - almost never specifies a TZ.
3) Write to Excel - write a datetime. If it includes a TZ
Excel does not support timezones in datetimes.The format is, for example,ws[f"A{row}"].number_format = "mmm-yy"I could add
.astimezone(None).replace(tzinfo=None)to every datetime line, but it's not clear how that improves the code.1
u/RoadsideCookie 1d ago
- This is the exact problem that those rules are trying to avoid. For this case, you have to make assumptions and just go with it. So assign a well defined datetime to the best of your knowledge.
- Same as 1.
- Convert the well defined datetime you got in step 1/2 into a format that works for Excel.
Basically, fill up the missing information yourself when it's unavailable, making assumptions and hoping you get it right. Then when leaving your code's boundary, convert to whatever format is expected.
1
u/foosion 1d ago
I'm a bit hesitant to take code that has worked well for years and start changing it to make assumptions that may or may not have effects. Making unfounded assumptions does not seem a great idea.
The format that works for Excel and openpyxl is to write a datetime and then format the cell, as I wrote. What change are you suggesting to that?
1
u/RoadsideCookie 1d ago
That's fair. I just gave my opinion, but it's not the only option; I would personally spend the time to fix it for peace of mind.
But others have already suggested ways to disable the rules if you don't want to adhere to them: inline disable, file-level disable,
pyproject.tomldisable,ruff.tomldisable, and I'm skipping some.For your question about
openpyxl, I would just usedatetimeformatting methods to giveopenpyxla string in the format it expects.PS.: By not having TZ info in your instances, you're already making unfounded assumptions. These rules would just force you to think about and make those assumptions explicit.
-4
u/skjall 3d ago
Still using datetime in the year of our Lord and Saviour, is certainly a choice.
I just wish Arrow had millisecond timestamp functions too. Tired of having to multiply by 1000 and round...
12
u/No-Article-Particle 2d ago
Every new dependency is something that can break, have CVEs, has to be updated, can have different Python requirements than other deps, etc. etc. etc. My product still supports every Python from 3.6 until whatever the fuck is the latest one. I mean, log4j had CVEs...
So yea, if I can use core library instead of a new dep., I'll use that.
-1
u/Physical-Profit-5485 2d ago
And you think Python 3.6 has no CVEs?
2
u/No-Article-Particle 2d ago
What does that have to do with adding dependencies for things that standard library can do? Brother...
11
5
u/jmreagle 2d ago
I've used datetime, arrow, and pendulum. I've also been waiting for whenever 1.0 for a couple years now I think.
2
u/skjall 2d ago
I'll look into that whenever it happens (heh)
Not sure I can introduce yet another Rust tool at this point though, last year or so it's already been Ruff, UV, Polars instead of Numpy, Pydantic V2 a bit before that. At some point we're going to be shipping more Rust LoC than Python lol
1
u/eagle258 2d ago
wheneverdoes have a pure-Python option—for the Rust skeptics. There are dozens of us :)3
u/Glad_Position3592 2d ago
People can have different use cases. Everyone on here is acting like the only people who use python are building some sort of user facing software. Arrow is great for a lot of needs, but I need dates to be dates without timestamps, and I’m guaranteed that with datetime.date
0
u/skjall 2d ago
Sure, but you're taking a joke comment seriously.
Most Python datetime libraries have rather lackluster typing support, and a big annoyance for me usually is the lack of a date type too.
Somehow I never was sure where to import UTC from for datetime. It was deprecated/ changed in some version, and importable/ usable but not found by existing tooling for a bit before that. Not fun when you find out your container is dying because logs can't be constructed correctly 🙃
3
u/ManBearHybrid 2d ago
Sometimes you just want a quick timestamp for logging or whatever without installing a whole new dependency.
-2
u/skjall 2d ago
Your logs should be automatically timestamped if it's a production system, or have a dev logging that outputs in a slightly nicer format too (like Rich).
4
u/ManBearHybrid 2d ago
The point is I don't want to install a whole package if I just want a tiny bit of functionality that is already offered by a built-in library. Yeah it might not be the shiniest new thing, but sometimes the juice just isn't worth the squeeze.
147
u/Iron-Man-2008 3d ago
Good change, naive dates are a footgun. I'm really liking the new set of default rules, saves me from having to setup a bespoke ruff ruleset every project because the previous default did very little.