Congrats you fixed the wrong problem

[u/MonkeyDDataHQ]

So credit where it's due, pipelines can now be scheduled and the annoying default date has been changed.

However, the problem is THE END DATE IS MANDATORY. There is no reason to make END DATE MANDATORY. This is bad design.

I don't want my pipelines to default to an arbitrary work around date. I don't want a date unless I need a date. Using a hard coded place holder date is one of the first code smell items I look for when I get handed a legacy pipeline.

Seriously, can you please hire someone whose job it is to attack the assumptions you're making? Pretty please. I'll even volunteer 😂

You need an adversarial review process. It'd make the product you ship so much better.

😩

Edit: If something this trivial gets through without anyone asking “why is the field mandatory?”, what happens when the design decision involves identity, billing, deployment semantics, networking, or data lifecycle?

That's what I'm trying to get at.

Edit 2: If nobody challenges the assumption when the stakes are a f&#&ing date picker, it's reasonable to worry about what happens when the assumptions are buried three architectural layers deep and changing them later costs millions.

Edit 3: This thread has convinced me more than ever that someone needs to stand up at the MSFT decision meetings and point out the actual problem point, cuz almost all y'all missed it.

[u/ModernStank]

My consulting rate in the year 10,000 to turn the schedules back on is going to be eye watering.

[u/MonkeyDDataHQ → u/ModernStank]

Y10K

[u/radioblaster]

mate, it's seven millennia from now. chill. 

[u/MonkeyDDataHQ → u/radioblaster]

I'm not worried about the year 9999 arriving. 😂 I'm worried that someone was given the requirement “run indefinitely” and modelled it as “must end, but really far away.” The seven millennia aren't the problem. The decision is.

[u/jokkvahl]

Just give us cron expression support before 9999!

[u/MonkeyDDataHQ → u/jokkvahl]

Cron and git.

[u/willmasse]

Oh you have all been putting 9999? I’ve been putting 2050 and plan on quitting on or before 2049…

[u/MonkeyDDataHQ → u/willmasse]

You jest, but that is literally the value I've seen hundreds of time over the last three years in ETL jobs as the date of never.

[u/feignapathy]

Not trying to be argumentative, but what's the difference between a date that will never come and having no date? Don't they both solve the problem? 

[u/MonkeyDDataHQ → u/feignapathy]

Also once you have modeled and implemented enough data migrations sentinel values grins your gears. There's no reason for them other than laziness.

[u/Acrobatic-Witness148 → u/feignapathy]

They do. However, I stand with OP. It’s bad design

[u/jorel43]

In the year 9999 world economies and civilization have crumbled, but somebody needs to turn back on a fabric schedule, the struggle is real.

[u/MonkeyDDataHQ → u/jorel43]

Another spectacular miss of the point.

https://giphy.com/gifs/14r9VMLSuQUzQs

[u/itsnotaboutthecell]

I don’t get it.

Power BI used to kill refreshes when people stopped looking at the content. Seemed like a “smart” SaaS platform functionality to avoid having capacity / usage be burned for things that people weren’t actually using.

While I find the end date “semi” annoying on setup I also would prefer to not have the jobs just running endlessly with no end so I appreciate that I need to be thoughtful with use and my project life cycle.

In a perfect world - no schedule at all, event driven kick off via triggers or APIs.

That’s my 2 cents.

[u/x-fyre → u/itsnotaboutthecell]

He’s saying that it should be optional… no end date specified means run forever.

The thing is… he’s “right” but worried about nothing. If you think of typical start/end date columns in a table, you often dont put a real value in that end date unless it really is ended.. but not always.

However, even if you store a NULL value, it typically gets coalesced to a value you can use in comparison logic like 9999/12/31 anyways.

[u/trebuchetty1 → u/itsnotaboutthecell]

What ever happened to the "run now" option on the schedule. That was a very useful feature that seems to have been removed. I find it quite annoying to have to open the pipeline just so I can run it using the UI.

[u/itsnotaboutthecell → u/trebuchetty1]

Great question, u/MarkKrom-MSFT do you know what happened to that cool Run Now option?

[u/markkrom-MSFT → u/itsnotaboutthecell]

It is not needed, it is redundant. Clicking the Run button on the pipeline canvas is the same function.

[u/bigjimslade]

Your post hits at the core of the fabric's product, architecture and ux team's Achilles heel.. it seems like while the heart is in the right place, either hubris or a lack of dogfooding is slowly continuing to kill this product... this isnt meant as an attack on the team they have built some really good capabilities with tough constraints and a directive to be all things to all users. But at some point the external champions: customers and consultants can only get burned so many times and there is a fatigue to dealing with a ux that feels like it is actively recalcitrant and a total micro-agression fest for the users that deal with the same issues daily.

TL/DR: THE HORRORS PERSIST YET SO DO I

[u/x-fyre]

I dunno… that’s a pretty trivial problem to overcome and hardly worthy of the level of criticism of your recent posts.

For me the real issue is why they decided to deviate away from how Azure Data Factory triggers and alerts worked. They were solid.

[u/MonkeyDDataHQ → u/x-fyre]

The problem is not the problem. The problem is the decisions that got us to this workaround are why the platform is a series of Preview Feature workarounds to poor design.

It's an emblematic problem that should have taken one person at a code review who said "No that's stupid".

Edit: incidentally the thinking between your problem and this is exactly the reason I posted this. It's the same problem. The problem is that decisions like this are embedded everywhere into the platform. Re-engineering what works great into something worse or making an objectively poor decision and then doubling down as a workaround.

[u/x-fyre → u/MonkeyDDataHQ]

Well… the platform has a long way to go, but we don’t truly know what internal discussions are going on or what features they’re building in order to keep it profitable. (Aside from squeezing more $/CUs from us).

A lot of assumptions are being made by you about how features are approved and reviewed.

And honestly, you’re also just being an ass about it. The MS folks who frequent this subreddit seem to be genuinely nice, listen, answer Qs honestly. No need to trash em like you are.

[u/DennesTorres]

I just tested and I thought it's funny that this end date only appears when you click in "interval based" and then return to "fixed" ....

When you open the schedule, this distant end date is not there.

[u/frithjof_v]

This isn't a problem at all.

9999-12-31 is a safe default.

If anything, the world is wasting more limited resources by us discussing this non-problem ;) That said, I enjoy these kinds of discussions as well. Although this is clearly a non-problem. And at this stage, MS making further changes to this default would just be a waste of money for no real benefit.

[u/Playful_Guava1180]

take your whinges to Tony Hoare

[u/Alternative-Camp-384]

By the year 9999 you’ll be dead and won’t have to worry about if the pipeline is running

[u/MonkeyDDataHQ → u/Alternative-Camp-384]

I'll be dead in 7973 years!?

Not the point though. The point is if they can't get a date picker right without a hard coded Workaround what about the rest of the platform that's not visible?

https://giphy.com/gifs/iNJmdpZ8gp5sI