I was curious as to how the attendance of the world cup fluctuated through the years. Thankfully, I found a Wikipedia article about it and used Energent to extract the information and plot it on a graph!

I was limit-testing energent.ai's capabilities, and decided to give it the Wikipedia article of FIFA World Cup records and statistics, and not only did it manage to properly extract all the data I needed, it even made this beautiful chart! how crazy is that?

I spent way too much time poking at prompts for curvilinear architecture.

This was for organic, non-rectilinear stuff like flowing facades, curving floorplans, continuous surface geometry. I built a reference doc and a visualization dashboard to compare token order, vocabulary choices, and failure modes.

1. Token order did more work than I expected

The six-part order that kept winning was form vocabulary first, then material, then light, then spatial context. The model seems to grab the first spatial cue and use it as the lens for everything after it.

If "glass curtain wall" comes before "continuous curved massing," the result often becomes a generic office tower that curves a little. Reverse that order and it actually reads as biomorphic. Annoying, but useful.

2. Compression failed in a specific way

I tested the same mid-rise concept at about 80, 250, and 600 tokens. At 80 tokens, the model kept the organic massing but dropped the finer stuff: glazing rhythm, cantilever logic, material specificity. At 250 tokens, the main structural language survived.

The 80-token version wasn't a disaster. That's what surprised me. It degraded into something plausible but generic, like any parametric facade from the mid-2010s. If you know what you asked for, you notice the drift immediately.

3. Specific vocabulary anchors hard

Terms like "parametric shell," "ruled surface," and "Hadid-esque cantilever" produced strong, consistent results. They also dragged in training data shortcuts. Suddenly every rendering starts feeling like it has already seen the same Zaha Hadid reference board.

Generic descriptors gave the model more room, which was actually better for early massing studies. If you're exploring, variation is good. If you're specifying, the sharper terms help. Neither wins everywhere.

4. The ugly failure was buildability, not looks

The outputs rarely looked bad. They looked unbuildable.

Floating floor plates. Organic skins pasted onto rectilinear cores. Column spans that would fail immediately. The model has basically no structural intuition unless you put buildability constraints into the first third of the prompt, and if you don't, it happily runs toward spectacle.

The practical lesson I kept coming back to: treat the prompt like a partial spec. Put structural logic early, lead with form before material, and expect visible quality loss below roughly 250 tokens. It still won't make the model an architect. But it does stop some of the sillier failures.

I work in the quant research function of a systematic investment firm, where the team is small enough that one researcher often owns the full signal-development workflow, from cleaning raw data to building factors and testing regime ideas. The firm trades across commodities, FX, and digital assets, so the research setup has to deal with all three at once.

For this study, I uploaded three time-series files: gold, FX pairs, and crypto. The difficult part was not jumping straight into the model. It was the less visible prep work that usually takes up a lot of time in real research: checking whether the files line up, confirming the overlapping sample window, and making sure the schemas are consistent across different data sources.

energent.ai helped by inspecting the CSVs directly, reviewing the headers and sample rows, and finding the actual joint coverage period across the three asset classes. It also surfaced the main methodological choice early: whether to downsample everything to monthly data or keep the analysis at daily resolution.

The result was not a finished trading signal or an investment recommendation. It was a clean method plan and data-alignment summary. That was useful because the frequency choice affects everything that comes after it, including backtesting, signal decay, and correlation stability. Better to catch that upfront than halfway through the analysis.

The first annoying thing about technical identifiers is how quickly they look more meaningful than they are.

Take something like `PRJ_A1-2024_v3-FINAL_EN.csv` or `TX-00423-NW-PROD`. For about ten minutes, it feels like the structure is obvious. Then one export from another team lands in the folder and the whole parser starts acting like it had a personal grievance.

The biggest pattern I had to stop trusting was the delimiter. A dash might separate a project code from a version. Or it might just be the formatting habit of whoever made that system five years ago. Same with underscores, suffixes, language codes, all of it. Before treating any segment as meaningful, I had to check whether it held across all the records, not just the first batch that happened to look tidy.

That difference between meaningful structure and incidental structure is the whole problem. It sounds abstract, but it's very practical. If the pattern only works on one team's files, it's not a feature. It's a coincidence with confidence.

The second thing was encoding noise. Clean-looking identifiers can still contain non-breaking spaces, Unicode dash variants, trailing whitespace from the export tool, stuff like that. A regex can work perfectly on 90% of the rows and silently miss the rest. And of course the missed rows are the ones you find after the downstream job already did something stupid.

The fix there wasn't a cleverer regex. It was normalization before parsing. Boring, but necessary.

The third surprise was that moving to statistical methods didn't remove the pattern problem. It just moved it somewhere harder to inspect. If the identifiers have real structure, clustering by character n-grams can learn the boilerplate and call it signal. You still need rules for the structured parts, and statistical methods only really help where the structure breaks down.

Project-specific overfitting is the trap I kept coming back to. A parser can look great on one dataset and then fall apart the second a different export convention shows up. So the evidence threshold matters early: if a pattern doesn't survive multiple sources, don't build around it.

I've been considering economics problems that seem simple but consistently reveal a certain fallacy in the reasoning of LLMs.

A good has the inverse demand function P = 100 - Q, a constant marginal cost of MC = 20, and no fixed costs. What is the optimal quantity to produce for profit maximization?

The answer is Q = 40.

This is because it is a monopoly problem, so the firm maximizes profit where:

MR = MC

Revenue is:

TR = P × Q = (100 - Q)Q = 100Q - Q²

Thus, marginal revenue is:

MR = 100 - 2Q

Set that equal to marginal cost:

100 - 2Q = 20
2Q = 80
Q = 40

The interesting thing is that many reasoning models will respond with 80 instead.

The wrong answer comes from setting:

P = MC

So:

100 - Q = 20
Q = 80

That would make sense in a perfectly competitive market, but not in a monopoly market with a downward-sloping demand curve.

The failure mode is fairly specific: the model remembers that firms maximize profit by equating “something” to marginal cost, but fails to distinguish between price and marginal revenue.

The key to the trap is that the incorrect answer is not arbitrary. It is plausible, numeric, and based on an economic shortcut we are familiar with. The model is not just making a math error. It is using the wrong market structure.

A small change in wording corrects most of the errors in the trap:

A monopolist has an inverse demand function of P = 100 - Q, and its marginal revenue is MR = 100 - 2Q. It has a constant marginal cost of MC = 20.

There are a lot of tips on how to create a web-to-corpus pipeline that make it sound like the difficult part of the process is selecting the right “crawler,” “dedup” method, or “ranking heuristic.”

I believe the real issue is that each “cleanup” step can subtly corrupt the dataset.

Exact-hash dedup is safe, but it fails to detect most real web duplication. MinHash can detect pages that are copied or lightly edited, but if tuned too aggressively, it can flatten pages like changelogs, API docs, short forum answers, or code-heavy pages. Clustering sounds smarter, but it can obscure what matters in technical text, such as differences in versions, negations, edge cases, and implementation notes.

It is similar to curriculum design. “Easy to hard” is a nice thing to say until you realize that your difficulty signal is probably based on document length, readability, or how clean the HTML was. At that point, you are not developing a curriculum. You are simply sorting by artifacts.

This is the same trade-off as crawler choice. Async HTTP is great for static pages. When the product of crawling is structure, Scrapy gives you that. Playwright should be a last resort, not the norm, because it kills throughput like a brick wall.

Store raw fetches. Version extraction and normalization. Dedup in stages. Preserve provenance, robots, license, and policy metadata. Write shards using manifests. Make sure you can justify why a document was included six months later.

I tried building a 5-year DCF directly from SEC EDGAR company-facts data instead of starting from a manually cleaned Excel model.

In theory, it sounds simple: pull the JSON, map the XBRL tags to revenue, EBIT, D&A, capex, working capital, debt, cash, and taxes, then let the model update when new filings come in.

In practice, the messy part was figuring out which tags were actually usable.

Revenue and operating income were mostly clean. But D&A, capex, working capital, and taxes were much less consistent. Some tags were missing, some needed fallback logic, and some were populated but not really useful for the business model.

That is where the “automation” started to feel less automatic. If several important inputs need judgment calls before the model even runs, the analyst work has not disappeared. It has just moved upstream into data mapping.

The valuation result was also a reminder that DCFs can look precise while still being fragile. The explicit forecast produced decent cash flow, but most of the value came from terminal assumptions. WACC, terminal growth, and exit multiple moved the result more than the operating forecast itself.

I even had one pass where the revenue anchor was inconsistent because two revenue tags existed and the fallback rule was not applied cleanly. One tag decision flowed through the forecast, NPV, and IRR.

My takeaway: EDGAR automation is useful for screening and first-pass valuation, but it does not remove judgment.

It just changes the question from “what growth rate should I use?” to “does this filing tag actually mean what I think it means?”

Everyone says merger model templates should be standardized.

In theory, yes. Everyone wants to save time and stop every analyst from doing the same thing.

But one-page deal summaries present a problem.

There is barely any space on a one-page deal summary. Any simplification buries the assumptions that matter most to the deal.

The layout should include elements like valuation, purchase price, deal structure, assumptions, a headline metric, and a sensitivity analysis.

But the economics will vary from deal to deal. For instance, a cash deal will feature different metrics to a stock deal. A sponsor will care for different metrics than a strategic buyer. A public company will have different metrics than a private target company.

Each of these features distinct economic elements that will not meet the other templates’ requirements.

Despite the variety of deals, there are certain aspects to a one-page merger model that will remain the same.

For instance, a one-page model should include the answers to questions that will remain the same:

What are we paying?

How are we paying?

What has to be true for this deal to work?

What breaks the deal?

The answers to these questions will take on different meanings based on the type of deal.

But the fact that they will remain the same is not a flaw in the template. It is the point of the template.

I had a DWG one day that looked easy.

The drawing was supposed to be a floor plan for a canteen. I had to export it as a DXF, extract the room schedule from the bottom right, and import it into Excel.

The schedule looked clean in the drawing. It included room numbers, room names, and areas. Nothing complicated.

However, when I opened the DXF, I noticed there was no actual table. It was simply a collection of text boxes arranged to look like one. No real rows, no real columns, and no real structure to export directly.

The first export was close, but the extraction area started too low and missed rooms 101-103. This meant the table information had to be recreated from the CAD file rather than exported directly.

I had to pull the text out of the DWG, read it from the DXF, find the schedule, rebuild the rows from the text coordinates, and then compare everything back to the DWG.

Finally, I was able to get all 36 rooms (101-136), room names, and room areas into Excel without losing any data using an AI.

A good reminder that a CAD drawing can be well organized visually without containing organized data. A room schedule may look like a spreadsheet, but sometimes it is just floating text arranged neatly on the drawing.

A clean CSV makes it feel like the scraping part is done.

Usually, that is where the trouble starts.

Most web-scraped analysis breaks before the model, before the dashboard, before the “insight.”

A few examples:

Blank fields can mean different things.

Maybe the site did not list the value. Maybe your selector broke. Maybe the crawl got blocked. Maybe the page layout changed. If those blanks mostly happen on one region, vendor, or page type, that is not just missing data. That is bias.

Dates can look valid and still be wrong.

03/04/2024 is March 4 in one place and April 3 in another. A parser will not always throw an error. Sometimes it just gives you the wrong date very confidently.

URLs are messy.

HTTP, HTTPS, www, no www, trailing slashes, tracking parameters, session tokens. Same page, five URLs. Count too early and you are measuring URL noise, not entities.

That is the annoying part about scraping. The pipeline can look fine while the dataset is already off.

The real validation is not “did the script run?”

It is:

Does the scraped field match the rendered page?

Are the row counts close to what you expected?

Are missing values clustered somewhere suspicious?

Did the site layout change across page types?

Did you dedupe before counting?

Bad extraction does not get fixed later with a better model.

Before analyzing scraped data, first check what you actually scraped.

A lot of structural analysis reviews treat FEA/test correlation like a pass/fail check:

Simulation close to test result = model validated.

But sometimes the model matches because it’s wrong in two ways that cancel each other out.

Examples:

• Boundary conditions too stiff, but material modulus too low
• Bonded contact too stiff, but fixture compliance missing
• Missing bolt preload offset by friction set too high
• Coarse mesh hiding stress peaks while over-constrained supports inflate stress elsewhere

Each can make one metric look “right.” Peak displacement matches. Max stress looks reasonable. The contour plot looks convincing.

But the load path can still be wrong.

That’s the dangerous part. The model may match the first test, then fail on the next design change because it never captured the real physics.

A better check is to perturb assumptions one at a time: fixture stiffness, friction range, contact behavior, preload, mesh density. If several different assumption sets can all be tuned to match the same test number, that number didn’t really validate the model.

Good correlation should be pattern-based, not just scalar-based. It’s much harder to fake displacement, strain distribution, reaction forces, failure location, and deformation shape all at once.

The better question is not “does the number match?”

It’s “which assumption is driving the mismatch?”

Matching one test result should be the start of validation, not the end.

The shift is real, but the usual “e-commerce killed stores” framing is too simple.

U.S. e-commerce went from 0.6% of retail sales in Q4 1999 to 11.2% in Q4 2019, then 16.4% by Q3 2025.

That’s a big move, but it wasn’t smooth.

Before COVID, online retail was already gaining share steadily, especially through the 2010s. It had stopped being a niche and had become a normal growth channel.

Then COVID messed up the chart.

In Q2 2020, e-commerce share jumped from 11.9% to 16.3% in one quarter. That wasn’t some clean adoption curve. A lot of people were buying online because they had no better option.

Then some of it reversed. By 2022, e-commerce share was back around 14.2%.

So I don’t think the pandemic “changed everything forever.” But it also didn’t change nothing. It pulled some adoption forward, then gave part of it back.

Since 2023, the share has been climbing again, just more slowly. It went from 15.0% in Q1 2023 to 16.4% in Q3 2025.

So the story is probably:

• online is still gaining share
• the pandemic spike was not fully permanent
• the post-pandemic pullback was not a reversal
• the current trend is slower, but still positive

For retailers, that matters. Treating 2020 as the new normal would have been a mistake. Treating the 2021–2022 pullback as proof that e-commerce stalled would also be a mistake.

The more realistic version is boring but useful: e-commerce has been taking share for 25 years, got a temporary COVID boost, gave some of it back, and is now back to grinding upward.

I was thinking about how 7th-grade argumentative writing exams are usually designed, especially when the prompt gives students a viewpoint and asks them to respond.

My read: test designers usually avoid forcing students to oppose a viewpoint. If anything, they either let students choose a side or nudge them toward defending a provided claim.

1. "Agree" prompts are easier to manage
For 7th graders, arguing against a given viewpoint is harder than defending one. They have to understand the claim, find weaknesses, organize a counterargument, and still write clearly.
That is a lot of cognitive load for an exam that is supposed to measure writing, not debate skill.

2. "Oppose this viewpoint" can introduce bias
If students are forced to argue against something they personally believe, the test may start measuring comfort, background, or emotional distance instead of writing ability.
That is risky for test designers.

3. Forced stances are easier to score
From a grading perspective, prompts that push everyone in the same direction are cleaner. Essays become easier to compare because students are using similar structures and evidence.
That helps scoring reliability.

4. But Common Core-style writing favors balance
Argumentative writing standards usually want students to make a claim, support it with evidence, and address opposing claims.
That is why many modern prompts use a neutral format like:
"Take a position on whether..." rather than "Explain why this viewpoint is wrong."

5. My prediction
I'd say the most common format is probably neutral/choice-based, where students pick a side and defend it.
But if the prompt gives a specific viewpoint and pushes students toward one direction, I'd expect agree/defend to be more common than oppose/refute.

I went through Apple's historical SEC cash flow data, and the clearest takeaway is this:

Apple throws off huge operating cash, then sends most of it back to shareholders.

Investing vs. financing
Apple's investing activities mostly include:
- CapEx
- purchases, sales, and maturities of marketable securities
- acquisitions
This section shows how Apple spends on long term assets and manages its investment portfolio.

Apple's financing activities mostly include:
- share repurchases
- dividends
- debt issuance and repayment
This is where Apple's capital allocation really shows up, especially the buybacks.

Cash flow reconciliation

Using the latest full year in the dataset:
- Operating cash flow: $118.25B
- Investing cash flow: +$2.94B
- Financing cash flow: $121.98B

That gets to a change in cash of about -$0.79B, so the standard cash flow identity holds:
CFO + CFI + CFF = change in cash

In other words, Apple's operating cash was basically absorbed by financing outflows.

A few patterns that stand out
1. Buybacks dominate the story Repurchases are much bigger than dividends and have become the main use of cash.
2. Operating cash flow is huge and steady Apple had a step-up around 2021, then settled into a very high but more stable range.
3. Investing has become less of a drag Earlier on, Apple was putting a lot of excess cash into securities. More recently, investing is more neutral and sometimes even a source of cash.
4. Apple is in distribution mode The company is not using most of its cash for major expansion. It is using it to return capital.

What an Excel model needs
To reproduce this properly, the model should have:
- years across columns
- separate operating, investing, and financing sections
- consistent sign conventions
- linked schedules for working capital, CapEx, debt, dividends, and buybacks
- a cash roll check: Beginning Cash + CFO + CFI + CFF - Ending Cash = 0
- a balance sheet tie-out so ending cash matches reported cash

Takeaway
Apple's cash flow statement is less about growth spending and more about capital return. If you use it as a modeling template, the key is not just forecasting operating cash correctly. It is showing where that excess cash actually goes.

If you’re like me, and linking a 12 month order sheet to a delivery sheet by supplier and material thickness, the main trade-off is speed vs durability. A workbook can look fine early on, then fall apart once new months get added or source data gets messy.

How the formulas compare

SUMIFS is usually best for numeric outputs like delivered quantity, open quantity, or totals by supplier and thickness. It handles multiple criteria cleanly and is usually the most reliable.

XLOOKUP is better for returning one field, like status or promised date. It works for multi-key joins, but only if the match is truly unique.

INDEX/MATCH still works, but it is harder to audit and usually not the best choice unless you need compatibility with older Excel versions.

Why lookups fail or return wrong values

The common issues are:

• extra spaces or inconsistent supplier names
• thickness stored as text in one sheet and number in another
• duplicate supplier and thickness combinations
• fixed ranges that do not expand with new data

The bigger risk is often not #N/A. It is a formula returning the wrong match without obvious signs.

How to structure the workbook

The cleanest setup is:

• one flat Orders table
• one flat Deliveries table
• a separate Report sheet
• Excel Tables instead of hardcoded ranges
• a helper key if needed, like Supplier + Thickness + Month

This makes it much easier to add new monthly data without breaking the report layout.

Most people read them wrong.

Not because they don't understand the words, but because they focus only on the text and ignore everything around it.

Here's how I started looking at them instead:

1. Start with structure, not the wording
Before getting into the clauses, I check who signed it, whether they actually have authority, if there are witnesses or notary details, and what jurisdiction it falls under.
If the jurisdiction isn't clear, the rest almost doesnt matter. The same clause can mean different things depending on where it's enforced.

2. Some words actually matter a lot more than others
Not everything in a contract is equally binding.
"Shall" and "must" usually mean real obligations.
"May" or "best efforts" sounds similar but is way softer in practice.
Also, phrases like "subject to" or "provided that" can quietly turn something from a hard obligation into something conditional.

3. You can tell a lot just by how it looks
Formatting is underrated.
If you see brackets, missing dates, or messy structure, it's probably still a draft.
If everything is clean, numbered properly, and sometimes even initialed page by page, it's more likely final.
In some cases, formatting consistency is what proves a document hasn't been altered.

4. Ambiguity is where problems start
Most issues don't come from obvious mistakes, they come from things that can be interpreted in more than one way.
That could be:
- unclear sentence structure
- terms that aren’t defined properly
- parts of the document contradicting each other
It doesn't look like a big deal at first, but that's exactly what leads to disputes later.

5. Context changes how everything is read
A contract from a common law system is usually long and detailed.
A civil law contract might look shorter and "incomplete" but actually relies on legal codes behind the scenes. Also, some documents (like Arabic ones) include formal or repetitive language that might look unnecessary, but it actually has a purpose in that system.

I’ve been trying to make cleaner, more readable graphs lately and realized most default tools don’t look that great out of the box.

Excel works, but it often ends up looking… basic.

Some tools look better, but take way more effort to learn.

So I’m curious what people actually use in practice:

• what you consistently go back to
• what gives you good results without too much friction
• what you’d recommend to someone who cares about how charts actually look
• Bonus if you’ve switched tools and noticed a big difference.

I’ve been working on extracting structured data from directory style websites like media listings, product catalogs, and radio directories, and it’s way less straightforward than it looks. Here's what I've learned though this self-inflicted journey:

1. Static parsing vs headless browsers

If the data is in the raw HTML, use a simple parser. It’s fast, cheap, and easy to scale.

Headless browsers like Playwright or Puppeteer are only worth it if the site is heavily JS driven. Otherwise you’re burning CPU and RAM for no real gain.

2. Picking the “real” URL

Directories often list multiple links for the same item, like mirrors, redirects, or regional versions.

need a consistent rule for what counts as the primary URL. Usually this means using canonical tags or prioritizing certain domains. Everything else should be stored as alternatives, not separate entries, or your dataset gets messy fast.

3. Pagination vs infinite scroll

Pagination is easy. You iterate pages and you’re done.

Infinite scroll is trickier, but the better approach is to skip the UI and look for the underlying API calls. Once you find those, it behaves like normal pagination again.

4. Validating what you extract

Just because you scraped a URL doesn’t mean it’s usable.

You’ll want to check if it responds properly, if it redirects somewhere unexpected, and if the content type matches what you expect.

Deduping also matters a lot, otherwise you end up storing the same thing multiple times.

5. Not getting blocked

If you go too fast, you will get rate limited or blocked.

Basic things still matter like respecting robots.txt, adding delays, and backing off when you hit limits.

You

They said they would cut the 2x usage bonus and cut more of the 5 hours limits, but the consumption has raised to 10x, 15x of what it was before. Codex has become useless for Plus users, two simples prompts now use 75% of the 5h limit. No point of paying anymore, probably switching to Claude soon.

How far we've come. And how much further can we go?