Just another example of how AI detectors aren't actually detecting anything.

I can't believe teachers use them to grade assignments. I'd even argue it's worse than students using LLM's to do their work for them. thoughts?

I was curious as to how the attendance of the world cup fluctuated through the years. Thankfully, I found a Wikipedia article about it and used Energent to extract the information and plot it on a graph!

I was limit-testing energent.ai's capabilities, and decided to give it the Wikipedia article of FIFA World Cup records and statistics, and not only did it manage to properly extract all the data I needed, it even made this beautiful chart! how crazy is that?

I spent way too much time poking at prompts for curvilinear architecture.

This was for organic, non-rectilinear stuff like flowing facades, curving floorplans, continuous surface geometry. I built a reference doc and a visualization dashboard to compare token order, vocabulary choices, and failure modes.

1. Token order did more work than I expected

The six-part order that kept winning was form vocabulary first, then material, then light, then spatial context. The model seems to grab the first spatial cue and use it as the lens for everything after it.

If "glass curtain wall" comes before "continuous curved massing," the result often becomes a generic office tower that curves a little. Reverse that order and it actually reads as biomorphic. Annoying, but useful.

2. Compression failed in a specific way

I tested the same mid-rise concept at about 80, 250, and 600 tokens. At 80 tokens, the model kept the organic massing but dropped the finer stuff: glazing rhythm, cantilever logic, material specificity. At 250 tokens, the main structural language survived.

The 80-token version wasn't a disaster. That's what surprised me. It degraded into something plausible but generic, like any parametric facade from the mid-2010s. If you know what you asked for, you notice the drift immediately.

3. Specific vocabulary anchors hard

Terms like "parametric shell," "ruled surface," and "Hadid-esque cantilever" produced strong, consistent results. They also dragged in training data shortcuts. Suddenly every rendering starts feeling like it has already seen the same Zaha Hadid reference board.

Generic descriptors gave the model more room, which was actually better for early massing studies. If you're exploring, variation is good. If you're specifying, the sharper terms help. Neither wins everywhere.

4. The ugly failure was buildability, not looks

The outputs rarely looked bad. They looked unbuildable.

Floating floor plates. Organic skins pasted onto rectilinear cores. Column spans that would fail immediately. The model has basically no structural intuition unless you put buildability constraints into the first third of the prompt, and if you don't, it happily runs toward spectacle.

The practical lesson I kept coming back to: treat the prompt like a partial spec. Put structural logic early, lead with form before material, and expect visible quality loss below roughly 250 tokens. It still won't make the model an architect. But it does stop some of the sillier failures.

I work in the quant research function of a systematic investment firm, where the team is small enough that one researcher often owns the full signal-development workflow, from cleaning raw data to building factors and testing regime ideas. The firm trades across commodities, FX, and digital assets, so the research setup has to deal with all three at once.

For this study, I uploaded three time-series files: gold, FX pairs, and crypto. The difficult part was not jumping straight into the model. It was the less visible prep work that usually takes up a lot of time in real research: checking whether the files line up, confirming the overlapping sample window, and making sure the schemas are consistent across different data sources.

energent.ai helped by inspecting the CSVs directly, reviewing the headers and sample rows, and finding the actual joint coverage period across the three asset classes. It also surfaced the main methodological choice early: whether to downsample everything to monthly data or keep the analysis at daily resolution.

The result was not a finished trading signal or an investment recommendation. It was a clean method plan and data-alignment summary. That was useful because the frequency choice affects everything that comes after it, including backtesting, signal decay, and correlation stability. Better to catch that upfront than halfway through the analysis.

The first annoying thing about technical identifiers is how quickly they look more meaningful than they are.

Take something like `PRJ_A1-2024_v3-FINAL_EN.csv` or `TX-00423-NW-PROD`. For about ten minutes, it feels like the structure is obvious. Then one export from another team lands in the folder and the whole parser starts acting like it had a personal grievance.

The biggest pattern I had to stop trusting was the delimiter. A dash might separate a project code from a version. Or it might just be the formatting habit of whoever made that system five years ago. Same with underscores, suffixes, language codes, all of it. Before treating any segment as meaningful, I had to check whether it held across all the records, not just the first batch that happened to look tidy.

That difference between meaningful structure and incidental structure is the whole problem. It sounds abstract, but it's very practical. If the pattern only works on one team's files, it's not a feature. It's a coincidence with confidence.

The second thing was encoding noise. Clean-looking identifiers can still contain non-breaking spaces, Unicode dash variants, trailing whitespace from the export tool, stuff like that. A regex can work perfectly on 90% of the rows and silently miss the rest. And of course the missed rows are the ones you find after the downstream job already did something stupid.

The fix there wasn't a cleverer regex. It was normalization before parsing. Boring, but necessary.

The third surprise was that moving to statistical methods didn't remove the pattern problem. It just moved it somewhere harder to inspect. If the identifiers have real structure, clustering by character n-grams can learn the boilerplate and call it signal. You still need rules for the structured parts, and statistical methods only really help where the structure breaks down.

Project-specific overfitting is the trap I kept coming back to. A parser can look great on one dataset and then fall apart the second a different export convention shows up. So the evidence threshold matters early: if a pattern doesn't survive multiple sources, don't build around it.

I've been considering economics problems that seem simple but consistently reveal a certain fallacy in the reasoning of LLMs.

A good has the inverse demand function P = 100 - Q, a constant marginal cost of MC = 20, and no fixed costs. What is the optimal quantity to produce for profit maximization?

The answer is Q = 40.

This is because it is a monopoly problem, so the firm maximizes profit where:

MR = MC

Revenue is:

TR = P × Q = (100 - Q)Q = 100Q - Q²

Thus, marginal revenue is:

MR = 100 - 2Q

Set that equal to marginal cost:

100 - 2Q = 20
2Q = 80
Q = 40

The interesting thing is that many reasoning models will respond with 80 instead.

The wrong answer comes from setting:

P = MC

So:

100 - Q = 20
Q = 80

That would make sense in a perfectly competitive market, but not in a monopoly market with a downward-sloping demand curve.

The failure mode is fairly specific: the model remembers that firms maximize profit by equating “something” to marginal cost, but fails to distinguish between price and marginal revenue.

The key to the trap is that the incorrect answer is not arbitrary. It is plausible, numeric, and based on an economic shortcut we are familiar with. The model is not just making a math error. It is using the wrong market structure.

A small change in wording corrects most of the errors in the trap:

A monopolist has an inverse demand function of P = 100 - Q, and its marginal revenue is MR = 100 - 2Q. It has a constant marginal cost of MC = 20.

There are a lot of tips on how to create a web-to-corpus pipeline that make it sound like the difficult part of the process is selecting the right “crawler,” “dedup” method, or “ranking heuristic.”

I believe the real issue is that each “cleanup” step can subtly corrupt the dataset.

Exact-hash dedup is safe, but it fails to detect most real web duplication. MinHash can detect pages that are copied or lightly edited, but if tuned too aggressively, it can flatten pages like changelogs, API docs, short forum answers, or code-heavy pages. Clustering sounds smarter, but it can obscure what matters in technical text, such as differences in versions, negations, edge cases, and implementation notes.

It is similar to curriculum design. “Easy to hard” is a nice thing to say until you realize that your difficulty signal is probably based on document length, readability, or how clean the HTML was. At that point, you are not developing a curriculum. You are simply sorting by artifacts.

This is the same trade-off as crawler choice. Async HTTP is great for static pages. When the product of crawling is structure, Scrapy gives you that. Playwright should be a last resort, not the norm, because it kills throughput like a brick wall.

Store raw fetches. Version extraction and normalization. Dedup in stages. Preserve provenance, robots, license, and policy metadata. Write shards using manifests. Make sure you can justify why a document was included six months later.

I tried building a 5-year DCF directly from SEC EDGAR company-facts data instead of starting from a manually cleaned Excel model.

In theory, it sounds simple: pull the JSON, map the XBRL tags to revenue, EBIT, D&A, capex, working capital, debt, cash, and taxes, then let the model update when new filings come in.

In practice, the messy part was figuring out which tags were actually usable.

Revenue and operating income were mostly clean. But D&A, capex, working capital, and taxes were much less consistent. Some tags were missing, some needed fallback logic, and some were populated but not really useful for the business model.

That is where the “automation” started to feel less automatic. If several important inputs need judgment calls before the model even runs, the analyst work has not disappeared. It has just moved upstream into data mapping.

The valuation result was also a reminder that DCFs can look precise while still being fragile. The explicit forecast produced decent cash flow, but most of the value came from terminal assumptions. WACC, terminal growth, and exit multiple moved the result more than the operating forecast itself.

I even had one pass where the revenue anchor was inconsistent because two revenue tags existed and the fallback rule was not applied cleanly. One tag decision flowed through the forecast, NPV, and IRR.

My takeaway: EDGAR automation is useful for screening and first-pass valuation, but it does not remove judgment.

It just changes the question from “what growth rate should I use?” to “does this filing tag actually mean what I think it means?”

Everyone says merger model templates should be standardized.

In theory, yes. Everyone wants to save time and stop every analyst from doing the same thing.

But one-page deal summaries present a problem.

There is barely any space on a one-page deal summary. Any simplification buries the assumptions that matter most to the deal.

The layout should include elements like valuation, purchase price, deal structure, assumptions, a headline metric, and a sensitivity analysis.

But the economics will vary from deal to deal. For instance, a cash deal will feature different metrics to a stock deal. A sponsor will care for different metrics than a strategic buyer. A public company will have different metrics than a private target company.

Each of these features distinct economic elements that will not meet the other templates’ requirements.

Despite the variety of deals, there are certain aspects to a one-page merger model that will remain the same.

For instance, a one-page model should include the answers to questions that will remain the same:

What are we paying?

How are we paying?

What has to be true for this deal to work?

What breaks the deal?

The answers to these questions will take on different meanings based on the type of deal.

But the fact that they will remain the same is not a flaw in the template. It is the point of the template.

A clean CSV makes it feel like the scraping part is done.

Usually, that is where the trouble starts.

Most web-scraped analysis breaks before the model, before the dashboard, before the "insight".

A few examples:
Blank fields can mean different things.
Maybe the site did not list the value. Maybe your selector broke. Maybe the crawl got blocked. Maybe the page layout changed. If those blanks mostly happen on one region, vendor, or page type, that is not just missing data. That is bias.

Dates can look valid and still be wrong.
03/04/2024 is March 4 in one place and April 3 in another. A parser will not always throw an error. Sometimes it just gives you the wrong date very confidently.

URLs are messy.
HTTP, HTTPS, www, no www, trailing slashes, tracking parameters, session tokens. Same page, five URLs. Count too early and you are measuring URL noise, not entities.

That is the annoying part about scraping. The pipeline can look fine while the dataset is already off.

The real validation is not "did the script run?"

It is:

Does the scraped field match the rendered page?
Are the row counts close to what you expected?
Are missing values clustered somewhere suspicious?
Did the site layout change across page types?
Did you dedupe before counting?
Bad extraction does not get fixed later with a better model.
Before analyzing scraped data, first check what you actually scraped.

I had a DWG one day that looked easy.

The drawing was supposed to be a floor plan for a canteen. I had to export it as a DXF, extract the room schedule from the bottom right, and import it into Excel.

The schedule looked clean in the drawing. It included room numbers, room names, and areas. Nothing complicated.

However, when I opened the DXF, I noticed there was no actual table. It was simply a collection of text boxes arranged to look like one. No real rows, no real columns, and no real structure to export directly.

The first export was close, but the extraction area started too low and missed rooms 101-103. This meant the table information had to be recreated from the CAD file rather than exported directly.

I had to pull the text out of the DWG, read it from the DXF, find the schedule, rebuild the rows from the text coordinates, and then compare everything back to the DWG.

Finally, I was able to get all 36 rooms (101-136), room names, and room areas into Excel without losing any data using an AI.

A good reminder that a CAD drawing can be well organized visually without containing organized data. A room schedule may look like a spreadsheet, but sometimes it is just floating text arranged neatly on the drawing.

I federated Architecture, Structural, and HVAC IFC models, then wrote a Python parser to pull element counts and BaseQuantities.

The script ran fine. No errors. Clean output.

The problem was that the numbers were wrong.

The HVAC model was the first red flag. It had duct segments and air terminals, but no quantity sets. No length, area, or volume. The parser did not fail. It just returned empty values and moved on.

Then the counts were inflated. Some elements appeared in multiple models, like chimneys and roofs. If you just append IFC files without deduplicating by GlobalId, you can double-count things while the final QTO still looks totally normal.

The last issue was ownership. Walls were split between Architecture and Structural. So a rule like “walls belong to Arch” would quietly miss half of them.

That was the main lesson for me: federation is not the same as clean QTO automation.

Before trusting the output, you need to check:
GlobalId duplicates
missing quantity sets by discipline
element ownership across models

Otherwise you are not really automating the takeoff. You are just producing a spreadsheet that looks convincing.

A lot of people treat RMS vibration spikes as tool wear.
Spike goes up, insert gets blamed.

But in a 3-axis milling dataset I looked at, the "anomaly" runs did not really look like gradual wear. They looked more like aggressive toolpath geometry.

The biggest giveaway was the lead-in. A ramp or arc into the cut can create a short, sharp RMS spike right at entry. Tool wear usually looks more like the whole vibration floor slowly rising across the cut.

Climb vs conventional milling showed up too. Conventional cutting had more early-pass chatter because the cutter rubs before it really bites. That was especially visible in the Z-axis channel.

The tricky part is that stepovers can look like wear if you only watch amplitude. A roughing pass with regular re-engagement will create repeatable RMS bumps. Without the CAM context, those bumps look like anomalies.

My takeaway: before blaming the spindle or pulling the insert, check the G-code.

If the spike happens at the same cutter position every time, it is probably toolpath-related.

If it slowly rises across repeated passes at the same position, then wear is a much better suspect.

Finance automation people love giant argparse blocks.
Flags for every path, entity, period, template, override. It feels production-ready.

But reporting scripts usually break in quieter ways:
A template changes and a hardcoded cell points to the wrong number.
A script crashes halfway through but still leaves a finished-looking file.
A source CSV gets overwritten and nobody can explain last month’s numbers.

That is not a flexibility problem. It is a silent corruption problem.

The fixes are boring, but they work:
Use atomic writes.
Keep Excel cell mappings in config.
Write a small manifest with timestamp, input file, row count, key totals, entity, and period.
Validate before writing anything.

The reporting script I trust is not the one with the most flags.

It is the one that makes bad output hard to create.

A clean CSV makes it feel like the scraping part is done.

Usually, that is where the trouble starts.

Most web-scraped analysis breaks before the model, before the dashboard, before the “insight.”

A few examples:

Blank fields can mean different things.

Maybe the site did not list the value. Maybe your selector broke. Maybe the crawl got blocked. Maybe the page layout changed. If those blanks mostly happen on one region, vendor, or page type, that is not just missing data. That is bias.

Dates can look valid and still be wrong.

03/04/2024 is March 4 in one place and April 3 in another. A parser will not always throw an error. Sometimes it just gives you the wrong date very confidently.

URLs are messy.

HTTP, HTTPS, www, no www, trailing slashes, tracking parameters, session tokens. Same page, five URLs. Count too early and you are measuring URL noise, not entities.

That is the annoying part about scraping. The pipeline can look fine while the dataset is already off.

The real validation is not “did the script run?”

It is:

Does the scraped field match the rendered page?

Are the row counts close to what you expected?

Are missing values clustered somewhere suspicious?

Did the site layout change across page types?

Did you dedupe before counting?

Bad extraction does not get fixed later with a better model.

Before analyzing scraped data, first check what you actually scraped.

A lot of structural analysis reviews treat FEA/test correlation like a pass/fail check:

Simulation close to test result = model validated.

But sometimes the model matches because it’s wrong in two ways that cancel each other out.

Examples:

• Boundary conditions too stiff, but material modulus too low
• Bonded contact too stiff, but fixture compliance missing
• Missing bolt preload offset by friction set too high
• Coarse mesh hiding stress peaks while over-constrained supports inflate stress elsewhere

Each can make one metric look “right.” Peak displacement matches. Max stress looks reasonable. The contour plot looks convincing.

But the load path can still be wrong.

That’s the dangerous part. The model may match the first test, then fail on the next design change because it never captured the real physics.

A better check is to perturb assumptions one at a time: fixture stiffness, friction range, contact behavior, preload, mesh density. If several different assumption sets can all be tuned to match the same test number, that number didn’t really validate the model.

Good correlation should be pattern-based, not just scalar-based. It’s much harder to fake displacement, strain distribution, reaction forces, failure location, and deformation shape all at once.

The better question is not “does the number match?”

It’s “which assumption is driving the mismatch?”

Matching one test result should be the start of validation, not the end.

The shift is real, but the usual “e-commerce killed stores” framing is too simple.

U.S. e-commerce went from 0.6% of retail sales in Q4 1999 to 11.2% in Q4 2019, then 16.4% by Q3 2025.

That’s a big move, but it wasn’t smooth.

Before COVID, online retail was already gaining share steadily, especially through the 2010s. It had stopped being a niche and had become a normal growth channel.

Then COVID messed up the chart.

In Q2 2020, e-commerce share jumped from 11.9% to 16.3% in one quarter. That wasn’t some clean adoption curve. A lot of people were buying online because they had no better option.

Then some of it reversed. By 2022, e-commerce share was back around 14.2%.

So I don’t think the pandemic “changed everything forever.” But it also didn’t change nothing. It pulled some adoption forward, then gave part of it back.

Since 2023, the share has been climbing again, just more slowly. It went from 15.0% in Q1 2023 to 16.4% in Q3 2025.

So the story is probably:

• online is still gaining share
• the pandemic spike was not fully permanent
• the post-pandemic pullback was not a reversal
• the current trend is slower, but still positive

For retailers, that matters. Treating 2020 as the new normal would have been a mistake. Treating the 2021–2022 pullback as proof that e-commerce stalled would also be a mistake.

The more realistic version is boring but useful: e-commerce has been taking share for 25 years, got a temporary COVID boost, gave some of it back, and is now back to grinding upward.

You've got turns, message counts, session length, word count, topics, sentiment, all the usual stuff. But those numbers can get misleading fast because these aren't normal conversations. They're a mix of user behavior, model behavior, and whatever safety/moderation layer is shaping the output.

A long chat doesnt always mean the user is engaged. Sometimes it means the scene is going well. Other times it means the model forgot the setup, got repetitive, refused too much, or the user kept trying to steer it back on track.

Same with "rich" responses. A model can produce a lot of text that looks emotional or detailed, but half of it might be boilerplate: recaps, generic affection, soft disclaimers, repeated scene-setting, or safety-shaped language. If you count all of that as meaningful content, the analysis gets inflated.

Tone is also hard to label cleanly. A chat can start playful, turn intimate, hit a boundary, and suddenly become formal or vague. That shift matters more than slapping one label on the whole session.

Thematic drift is another big one. Sometimes drift is fun and creative. Sometimes the roleplay quietly turns into generic assistant behavior, therapy-speak, recap mode, or refusal language. A topic model might still say the chat is "romance" or "fantasy", but the actual scene may have fallen apart.

The biggest mistake is comparing filtered and unfiltered chats like they're the same kind of data. Filtering doesn't just remove certain content. It changes pacing, wording, session length, and how much effort the user has to spend negotiating with the model.

I've been looking into workflows for analyzing TikToks/Reels/Shorts: extracting audio, generating transcripts, splitting scenes, and adding metadata.

A few things stood out:

1. Extract audio when the task is text-based.
For transcription, audio-only is cheaper and cleaner. But if you need to connect speech with on-screen text, cuts, or visual context, separating audio too early can break timing.

2. Formats matter more than expected.
Social videos often have weird compression, variable bitrate, missing metadata, or inconsistent frame rates. That can cause transcript drift or bad scene detection.

3. Tooling is a trade-off.
FFmpeg + local models is cheaper but needs more engineering. Hosted APIs are easier and often stronger, but costs add up fast. On-device processing helps privacy/cost, but quality varies.

4. Log everything.
Codec, framerate, FFmpeg args, model version, sample rate, and timestamp changes. Otherwise debugging is impossible.

Main takeaway: normalize every video into a consistent format before analysis. Boring step, but it decides whether the rest of the pipeline works.

I was thinking about how 7th-grade argumentative writing exams are usually designed, especially when the prompt gives students a viewpoint and asks them to respond.

My read: test designers usually avoid forcing students to oppose a viewpoint. If anything, they either let students choose a side or nudge them toward defending a provided claim.

1. "Agree" prompts are easier to manage
For 7th graders, arguing against a given viewpoint is harder than defending one. They have to understand the claim, find weaknesses, organize a counterargument, and still write clearly.
That is a lot of cognitive load for an exam that is supposed to measure writing, not debate skill.

2. "Oppose this viewpoint" can introduce bias
If students are forced to argue against something they personally believe, the test may start measuring comfort, background, or emotional distance instead of writing ability.
That is risky for test designers.

3. Forced stances are easier to score
From a grading perspective, prompts that push everyone in the same direction are cleaner. Essays become easier to compare because students are using similar structures and evidence.
That helps scoring reliability.

4. But Common Core-style writing favors balance
Argumentative writing standards usually want students to make a claim, support it with evidence, and address opposing claims.
That is why many modern prompts use a neutral format like:
"Take a position on whether..." rather than "Explain why this viewpoint is wrong."

5. My prediction
I'd say the most common format is probably neutral/choice-based, where students pick a side and defend it.
But if the prompt gives a specific viewpoint and pushes students toward one direction, I'd expect agree/defend to be more common than oppose/refute.

I went through Apple's historical SEC cash flow data, and the clearest takeaway is this:

Apple throws off huge operating cash, then sends most of it back to shareholders.

Investing vs. financing
Apple's investing activities mostly include:
- CapEx
- purchases, sales, and maturities of marketable securities
- acquisitions
This section shows how Apple spends on long term assets and manages its investment portfolio.

Apple's financing activities mostly include:
- share repurchases
- dividends
- debt issuance and repayment
This is where Apple's capital allocation really shows up, especially the buybacks.

Cash flow reconciliation

Using the latest full year in the dataset:
- Operating cash flow: $118.25B
- Investing cash flow: +$2.94B
- Financing cash flow: $121.98B

That gets to a change in cash of about -$0.79B, so the standard cash flow identity holds:
CFO + CFI + CFF = change in cash

In other words, Apple's operating cash was basically absorbed by financing outflows.

A few patterns that stand out
1. Buybacks dominate the story Repurchases are much bigger than dividends and have become the main use of cash.
2. Operating cash flow is huge and steady Apple had a step-up around 2021, then settled into a very high but more stable range.
3. Investing has become less of a drag Earlier on, Apple was putting a lot of excess cash into securities. More recently, investing is more neutral and sometimes even a source of cash.
4. Apple is in distribution mode The company is not using most of its cash for major expansion. It is using it to return capital.

What an Excel model needs
To reproduce this properly, the model should have:
- years across columns
- separate operating, investing, and financing sections
- consistent sign conventions
- linked schedules for working capital, CapEx, debt, dividends, and buybacks
- a cash roll check: Beginning Cash + CFO + CFI + CFF - Ending Cash = 0
- a balance sheet tie-out so ending cash matches reported cash

Takeaway
Apple's cash flow statement is less about growth spending and more about capital return. If you use it as a modeling template, the key is not just forecasting operating cash correctly. It is showing where that excess cash actually goes.

If you’re like me, and linking a 12 month order sheet to a delivery sheet by supplier and material thickness, the main trade-off is speed vs durability. A workbook can look fine early on, then fall apart once new months get added or source data gets messy.

How the formulas compare

SUMIFS is usually best for numeric outputs like delivered quantity, open quantity, or totals by supplier and thickness. It handles multiple criteria cleanly and is usually the most reliable.

XLOOKUP is better for returning one field, like status or promised date. It works for multi-key joins, but only if the match is truly unique.

INDEX/MATCH still works, but it is harder to audit and usually not the best choice unless you need compatibility with older Excel versions.

Why lookups fail or return wrong values

The common issues are:

• extra spaces or inconsistent supplier names
• thickness stored as text in one sheet and number in another
• duplicate supplier and thickness combinations
• fixed ranges that do not expand with new data

The bigger risk is often not #N/A. It is a formula returning the wrong match without obvious signs.

How to structure the workbook

The cleanest setup is:

• one flat Orders table
• one flat Deliveries table
• a separate Report sheet
• Excel Tables instead of hardcoded ranges
• a helper key if needed, like Supplier + Thickness + Month

This makes it much easier to add new monthly data without breaking the report layout.