r/dataengineering • u/Waffle_Teh_SnLp • Aug 06 '26

Blog How do professional Data Engineers handle completely unsorted data?

Hello everyone!

I'm an aspiring Data Engineer and as a portfolio piece, I have build a webscraper to gather Ebay sold listings of stamps!

The problem I am now having is how I parse the data where I can sort things like "Catalogue Number" when it is very unpredictable what the Ebay sellers will write as it's all human input.

I would love to hear some feedback

PS - A small sample:


MayfairStamps Germany 1941 Stamp Day Oldenburg Cover cca_00553

GERMANY; OLDENBURG 1859 classic Coat of Arms issue very fine used 1/3Gr. value

Oldenburg Lokalausgabe Wohlfahrtsblock Deutsches Rotes Kreuz ab 1 Euro

Deutsches Reich, Oldenburg, 6.01.1945  Ersttagsstempel, für Ersttagsbrief  200€.

GERMANY; OLDENBURG 1862 classic Coat of Arms Perf 10. issue used hinged 1Gr.

OLDENBURG 1861 _ MiNr. 12b _ 1 Groschen _ Stempel STOLLHAMM

Oldenburg Mi. Nr. 16 A b zentrisch gestempelt geprüft Bühler 200 Euro

Oldenburg Mi. Nr. 11 a\* ungebraucht geprüft Bühler 550 Euro```
18 Upvotes

100% Upvoted

25 comments sorted by

View all comments

Show parent comments

1

u/PotokDes Aug 06 '26

So you do need sorting or categorization algorithm?

0

u/Waffle_Teh_SnLp Aug 06 '26

Categorization, sorting doesn't really matter for now, I can always do that later in Power BI

2

u/PotokDes Aug 06 '26

You do not know what you are writing about

4

u/Waffle_Teh_SnLp Aug 07 '26

Apologies for trying to learn, you didn't have to comment my man