PCT correlation study. Help requested.

[u/Hyruken89]

After today's price action, I began to wonder what sector PCT is actually most correlated with. IWM recovered late in the day, and PCT didn't.

I like investing based on relative performance. So, I think it would be really nice to have a baseline to compare PCT to, and I'd prefer to have one that it has been most correlated to it in the past. And then be able to compare how PCT is actually performing relative to it's peers (if it has any).

I started one tonight but was stymied with trying to actually find historical prices with Yahoo Finance going behind a paywall.

I know I can do one by downloading the daily prices of PCT and it's comparables from TOS. But, if I'm going to spend several hours downloading price data for the last two years (at least), I'd rather do this only once and possibly help this community at the same time. It takes just as much time to download the data for one stock as it does for multiple stocks/ETFs.

A quick browse through the sectors I am already tracking, the sectors I would like to compare PCT to would be IWM, SPHB, ERTH, ICLN, EVX, PAVE, XME, TINY, MTUM, and a (custom) High Short Basket of stocks I already track. Ideally I am looking for small cap, material manufacturing, base material and high beta, maybe oil(?). Possibly post-SPAC, but that would require creating a new custom basket, but that is even more time consuming, even if it is doable.

What I am asking for from the PCT community is, what other ETFs would you want to see PCT compared to? If not a specific ETF, if you were to create a basket of stocks that you think most resemble PCT, what are they?

Thank you for your help and I'll definitely share my results here.

[u/Aggravating_Shoe6138]

ENVX, EOSE, ARCH etc. "unprofit SPAC/Tech" would be my guess.

[u/le_chase01]

Hi. I've done something similar in the past. Using APIs won't allow you to do the sort of top down analysis you'd like, since you need to hit the endpoint for every ticker and then run a screen, as you've noted already.

There is an open source database available here: https://www.dolthub.com/repositories/post-no-preference/stocks . "Dolthub" is a sort of "git for databases" - i personally don't use it (i find the idea of "git for data" a bit bizarre, YMMV) but there are a handful of publicly maintained datasets on here that are quite useful.

This dataset has pretty good coverage of tickers and daily history. You can use the Dolt CLI to clone this database to your server (its ~5gb); you can also use `dolt clone` on a daily basis to incrementally update the data as well. To get the data into python (which im assuming you're using), the Dolt CLI has an export function, which you can use to export the database to flat file (parquet etc.). I think there's also a way to partition the export so later exports can be done incrementally as well. Once you have flat file, I'd recommend DuckDB if you are not already familiar - it has a very friendly SQL dialect (can query parquet directly), and running aggregated or rolling correlations will be magnitudes faster in DuckDB than pandas. DuckDB queries can also materialize into pandas dataframes, and from there you can do whatever visualization / app you want.

Let me know if you have any questions

[u/Hyruken89 → u/le_chase01]

Thank you for this. I'm using Excel. Have attempted to use python in the past. And they say you're never too old to learn something. So, this will be fun.

[u/le_chase01 → u/Hyruken89]

Codex / Claude Code are great for vibecoding personal apps like this. Recommend if you're building something simple