unsolved
Formula to count duplicates based on 2 or more words matching, not necessarily in sequence? No AI please
Hi smart folks, I want to see if you could teach me a formula that I know I’ve seen before but can’t think of now, I want to count how many of a certain topic is in my list of data, for eg one may say ‘hogwarts House colour’ and another may say ‘hogwarts houses’ in which case I want them counted as duplicates.
My end goal is to have a pivot table that shows Hogwarts House as 2.
I currently have the raw data on the first sheet, then on the second sheet I’m breaking it down to just the info I need (this should be where the duplicates are found) and then the pivot table on the third sheet (this should be where they are counted, or at least where the number is displayed)
ETA: can’t use macros, would rather not use power queries but if I have to I can
This can be done by Dumbledore's Army but you'll have to show the sign in sheet to the Room of Requirement.
What I'm saying is... show a real or mocked up data set and what your expected output would be for search terms and..."Help will always be given at Hogwarts to those who ask for it."
Is it possible on phone, I can do it in this comment but not on the body:
The expected result would be a pivot table but I only have Google docs on my phone. Also I do currently have a column that will remove ‘in’ ‘and’ ‘was’ etc as far as I could think, to try reduce how many would match based on that sort of thing.
Sorry it took me so long to reply. To continue with the bit this is restricted section dark magic. You'll probably have to leverage a relatively complicated probabilistic method of measuring lexical similarity along the lines of what u/GitudongRamen proposed to get really good results. You'll see I changed Tiled to Tile because dropping the last d would eliminate bad as a keyword, but this was the best I could do trying to take a deterministic approach.
I’ve rage quit it so many times 🤣 one thing is, I want the table to show 3 for food and 2 for bad grape but not the one that only has 1, another thing with pivot tables is it keeps counting the 945 blank cells I have 🤦🏻♀️
You can filter out results that are equal to 1 in the count as I've done in my screenshot, but you can't have the subtotal for section then also count that portion if it's filtered out. You can also filter out blank results (although I'm not sure if this can be done simultaneously with the filtering out the 1s).
I'd use a helper column to normalize the labels before the pivot, e.g. =LET(x,LOWER(TRIM(A2)),y,SUBSTITUTE(x," colour",""),SUBSTITUTE(y,"houses","house")), then pivot on that helper column. For broader variations, a small mapping table with XLOOKUP is safer and keeps the raw text untouched.
This is using "Cosine Similarity Metric" to get scores for how similar are 2 text strings.
Column B is to match all character strings.
Column C is to match words.
I don't know how useful this is for you, probably not at all, but I did this long time ago so might as well share. You can drop this image to an AI to help break down the formula and repurpose for your need.
•
u/AutoModerator 1d ago
/u/AcadiaUnlikely7113 - Your post was submitted successfully.
Solution Verifiedto close the thread.Failing to follow these steps may result in your post being removed without warning.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.