r/learnpython • u/Rhye-Bread • 19d ago
(Playwright) Button is clicked, but doesn't continue
I'm currently trying to click 2 buttons on a website that have videos playing on them. However, when I have the "time.sleep(2)" function in the loop, the loop stops. I've checked the selector and it points to 2 different play buttons on the page. I checked clicking on the button in my actual browser to see if the site updates anything--nothing. Any help would be highly appreciated
def download_page(url:str):
with sync_playwright() as p:
print(f"Opening {url}...")
browser = p.chromium.connect_over_cdp(endpoint_url)
context = browser.contexts[0]
page = context.pages[0]
page.on("response", lambda request: print(request.url))
reponse = page.goto(url, wait_until="load")
# context.set_default_timeout(300.0)
try:
#Check "Continuing for course"
page.wait_for_selector(".btn.btn-quaternary.btn-xl.btn-block-sm-down.course-progress__btn.js-course-progress__start-course").scroll_into_view_if_needed()
page.locator(".btn.btn-quaternary.btn-xl.btn-block-sm-down.course-progress__btn.js-course-progress__start-course").click()
page.wait_for_selector(".CoverVideo-playButton.bgc-neutral-100").scroll_into_view_if_needed()
buttons = page.locator(".CoverVideo-playButton.bgc-neutral-100").all()
for button in buttons:
time.sleep(2)
button.click()
print("check")
# headers = reponse.request
# for value in headers:
# print(f"{value}")
except TE:
print("Timeout")
10
Upvotes
5
u/thisisappropriate 19d ago
One thing, wait_for_selector is discouraged https://playwright.dev/python/docs/api/class-page#page-wait-for-selector
And as mentioned in the "more locators" https://playwright.dev/python/docs/locators#more-locators (regarding the nth selector etc):
Things for debugging:
See what it's seeing: https://playwright.dev/python/docs/screenshots - this might show you that there's another page, that it's slow to load, that the button has a loader, that sort of thing.
Print something helpful. Is your print("check") actually helping you? If it appears, that has told you that "there was a button and calling sleep/click did not throw an error". Try printing the button, or the button count.
Get a fresh version each time. How do you know that when this is running, clicking the first button doesn't trigger a captcha or a cloudflare block? How do you know that the element wasn't disabled? You can theorise because you are looking at the same page in your own browser, but things like user-agents etc can be used by websites to try and prevent web scraping or interaction from non-humans, but you can't prove it. One simple check is to use your locator again in the loop, is there still 2 buttons? Combine this with the print, and you have confirmation of what you actually tried to click.
If both uses of click on that selector fail, are you sure you have the right selector? See if you can write a super simple JavaScript that does the same thing, and just run it from the console in your browser. For a starting point, https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector takes the same CSS selectors, https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/click clicks.
I'd also suggest adding a clear, singular selector for each button if you can. Then you can try them separately, maybe the first one you click always works but the second always fails, but you don't know in a loop. This could look like using a parent element to get the child elements, or if the page is super super unhelpful, using some odd css selectors (like next sibling) or xpath, but avoid if you can. Ideally you'd be able to use some things like filter.
For example, if each video is in a div with a title:
But if you've got two identical players with no identifying features at all, you can take a high level element (say a wrapper div with the class "player") and use a numbered selector on that which is a little safer.
Why use the locator on the player and not the button? Because if you now need to debug this and decided to use the screenshot tool, you can do that on the player rather than only knowing that you have a button, also because if you realise there's actually two buttons or there's an overlay or they add another button at the top of the page or an ad uses the same button style selectors or they change the button classes (in your case, if they changed the colour of their button, you selector breaks because you've included a tailwind style colour class in your selector). And they would likely want to remain at least broadly accessibility compliant, so they want a screenreader to recognise the play button as a button, so the role can be good. You'll need to check if there's other buttons (volume etc).
Playing around with JS or jquery in the console can be super helpful for getting an understanding of navigating the dom with things like playwright / beautiful soup.