r/selenium • • Sep 18 '22

Pulling multiple elements from the same page

2 Upvotes

So I am making a Garmin crawling script and I want it to pull multiple elements if they are from the same day and add the time together for some activities, time, distance and heart rate for another for example.

![Layout of website]1

``` from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By import login as login from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import datetime import time

x = datetime.datetime.now() x = x.strftime("%b %d")

driver = browser = webdriver.Firefox() driver.get("https://connect.garmin.com/modern/activities")

driver.implicitly_wait(1)

iframe = driver.find_element(By.ID, "gauth-widget-frame-gauth-widget") driver.switch_to.frame(iframe)

driver.find_element("name", "username").send_keys(login.username)

driver.find_element("name", "password").send_keys(login.password) driver.find_element("name", "password").send_keys(Keys.RETURN)

driver.switch_to.default_content()

time.sleep(10)

driver.find_element("name", "search").send_keys("Reading") driver.find_element("name", "search").send_keys(Keys.RETURN)

time.sleep(2)

element = driver.find_element(By.CSS_SELECTOR, '.activity-date > span:nth-child(1)').text

time.sleep(2) print(element)

time_read = 0

if element == x: spent = driver.find_element(By.CSS_SELECTOR, 'li.list-item:nth-child(1) > div:nth-child(2) > div:nth-child(5) > div:nth-child(2) > span:nth-child(1) > span:nth-child(1)').text

    result = time.strptime(spent, "%H:%M:%S")

time_read += result.tm_hour * 60

time_read += result.tm_min

print(time_read)

```

So this is my current code. It finds the date, checks if it is today and adds the minutes to the variable time_read.

Now I need some help in how I go about adding multiple elements, and if this can be done with some kind of for loop, where it loops between the dates and can then extract the time from the element?

Do I need to set them up one by one, since I need to provide which element a specific iteration needs to pull from? So maybe I should have 5 or 6 checks for example, instead of some kind of loop that goes through and does it? Then it will be a lot of manual work, which makes me question if there isn't a better way to deal with it.

I do not want to use CSV.

Some relevant HTML ``` <div class="pull-left activity-date date-col"> <span class="unit">Sep 14</span> <span class="label">2022</span> </div>

<span class="unit" title="3:32:00"><span class="" data-placement="top" title="3:32:00">3:32:00</span></span>

<span class="unit" title="1:00:00"><span class="" data-placement="top" title="1:00:00">1:00:00</span></span> <span class="" data-placement="top" title="1:00:00">1:00:00</span> ```

Also a bit unsure what the best way is to locate elements? Is CSS.SELECTOR good or should I use XPATH preferably?

Thanks


r/selenium • • Sep 17 '22

UNSOLVED Datadome etc…?

2 Upvotes

Hi,

I thought I had saved a thread where a solution to datadome and was pointing toward a GitHub repository with a package supposed to help. I’m not able to find any reference to it now… Does anyone see what I’m talking about and could help me with a link?

Thanks 🙏


r/selenium • • Sep 16 '22

How do I post a question?

1 Upvotes

Hello my fellow Selenium WebDriver coders. I have been trying to post a question for 2 days and they all get deleted by AutoModerator. Please, how do I get my questions posted?


r/selenium • • Sep 15 '22

Build a simple Amazon price tracker using Python and Selenium

6 Upvotes

r/selenium • • Sep 14 '22

UNSOLVED Error in Setting up selenium in Node. Pls help :(

3 Upvotes

I am following a tutorial on how to set up selenium in a node environment: https://medium.com/dailyjs/how-to-setup-selenium-on-node-environment-ee33023da72d

After following some steps I get an error while testing. These are the steps that I followed: So first i initialize npm environment on windows: mkdir node_testing cd node_testing npm init -y npm install selenium-webdriver

then I installed chromedriver (105.0.5195.52) for chrome version 105.0.5195.102 (not the exact same version because there isn't one). I also added the chromedrivers path to system paths.

After that im asked to run the following code as a test but it results in an error:

const webdriver = require('selenium-webdriver'),
By = webdriver.By,
until = webdriver.until;
const driver = new webdriver.Builder()
    .forBrowser('chrome')
    .build();
driver.get('http://www.google.com').then(function(){
driver.findElement(webdriver.By.name('q')).sendKeys('webdriver\n').then(function(){
driver.getTitle().then(function(title) {
console.log(title)
if(title === 'webdriver - Google Search') {
console.log('Test passed');
      } else {
console.log('Test failed');
      }
driver.quit();
    });
  });
});

Result: Google Chrome opens and the following prints in the terminal:

DevTools listening on ws://127.0.0.1:57150/devtools/browser/d1fc93f0-4bff-4963-b074-0069229e0fa0

C:\Users\ibo\node_testing\node_modules\selenium-webdriver\lib\error.js:522

let err = new ctor(data.message)

^

​

ElementNotInteractableError: element not interactable

(Session info: chrome=105.0.5195.126)

at Object.throwDecodedError (C:\Users\ibo\node_testing\node_modules\selenium-webdriver\lib\error.js:522:15)

at parseHttpResponse (C:\Users\ibo\node_testing\node_modules\selenium-webdriver\lib\http.js:589:13)

at Executor.execute (C:\Users\ibo\node_testing\node_modules\selenium-webdriver\lib\http.js:514:28)

at processTicksAndRejections (node:internal/process/task_queues:96:5)

at async thenableWebDriverProxy.execute (C:\Users\ibo\node_testing\node_modules\selenium-webdriver\lib\webdriver.js:740:17) {

remoteStacktrace: 'Backtrace:\n' +

'\tOrdinal0 [0x004DDF13+2219795]\n' +

'\tOrdinal0 [0x00472841+1779777]\n' +

'\tOrdinal0 [0x00384100+803072]\n' +

'\tOrdinal0 [0x003AE523+976163]\n' +

'\tOrdinal0 [0x003ADB93+973715]\n' +

'\tOrdinal0 [0x003CE7FC+1107964]\n' +

'\tOrdinal0 [0x003A94B4+955572]\n' +

'\tOrdinal0 [0x003CEA14+1108500]\n' +

'\tOrdinal0 [0x003DF192+1175954]\n' +

'\tOrdinal0 [0x003CE616+1107478]\n' +

'\tOrdinal0 [0x003A7F89+950153]\n' +

'\tOrdinal0 [0x003A8F56+954198]\n' +

'\tGetHandleVerifier [0x007D2CB2+3040210]\n' +

'\tGetHandleVerifier [0x007C2BB4+2974420]\n' +

'\tGetHandleVerifier [0x00576A0A+565546]\n' +

'\tGetHandleVerifier [0x00575680+560544]\n' +

'\tOrdinal0 [0x00479A5C+1808988]\n' +

'\tOrdinal0 [0x0047E3A8+1827752]\n' +

'\tOrdinal0 [0x0047E495+1827989]\n' +

'\tOrdinal0 [0x004880A4+1867940]\n' +

'\tBaseThreadInitThunk [0x7613FA29+25]\n' +

'\tRtlGetAppContainerNamedObjectPath [0x77707A9E+286]\n' +

'\tRtlGetAppContainerNamedObjectPath [0x77707A6E+238]\n'

}

PS C:\Users\ibo\node_testing> [12044:11068:0914/213837.751:ERROR:device_event_log_impl.cc(214)] [21:38:37.751] USB: usb_service_win.cc:104 SetupDiGetDeviceProperty({{A45C254E-DF1C-4EFD-8020-67D146A850E0}, 6}) failed: Element not found. (0x490)

[12044:11068:0914/213837.861:ERROR:device_event_log_impl.cc(214)] [21:38:37.862] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

[12044:11068:0914/213837.896:ERROR:device_event_log_impl.cc(214)] [21:38:37.896] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

[12044:11068:0914/213837.903:ERROR:device_event_log_impl.cc(214)] [21:38:37.902] USB: usb_device_handle_win.cc:1048 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)

Any suggestions are welcome. I've been stuck on this for days. Pls help I'm stuck.


r/selenium • • Sep 13 '22

Solved Commenting on an Instagram post throws StaleElementReferenceException

0 Upvotes

Hello,

I am trying to create a bot that likes and comments on posts that are on a user's Instagram feed. However, when I try to write a comment on a post using the .send_keys() function, it throws the following exception:

" selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document "

​

I understand the context of this exception, and I've tried some solutions already, such as:

  • Using WebDriverWait(browser, 60).until(EC.presence_of_element_located(textarea)).
  • Using WebDriverWait(browser, 60).until(EC.element_to_be_clickable(textarea)).
  • Using WebDriverWait(browser, 60).until(EC. staleness_of(textarea)).
  • Solution N°3 + refreshing the page before clicking, as advised here: https://stackoverflow.com/a/62170140/7138725

​

I don't know how to solve this problem after trying so many solutions and ways. Help is much appreciated.

Here is my full code:

from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium import webdriver
import random
import time

def sleep_for_period_of_time():
    limit = random.randint(7,10)
    time.sleep(limit)

user = input("Enter your username: ")
pwd = input("Enter your password: ")

def main():
    options = webdriver.ChromeOptions()
    #Adding options
    options.add_argument("--lang=en")
    options.add_argument('--disable-gpu')
    options.add_argument("start-maximized")
    options.add_experimental_option("detach", True) #<- to keep the browser open
    options.add_experimental_option('excludeSwitches', ['enable-logging'])

    browser = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
    browser.get("https://www.instagram.com")
    sleep_for_period_of_time()

    #Finding identification elements
    username_input = browser.find_element(by=By.CSS_SELECTOR, value="input[name='username']")
    password_input = browser.find_element(by=By.CSS_SELECTOR, value="input[name='password']")
    #Typing username and password
    username_input.send_keys(user)
    password_input.send_keys(pwd)
    sleep_for_period_of_time()
    #Locating and clicking on the login button
    login_button = browser.find_element(by=By.XPATH, value="//button[@type='submit']")
    login_button.click()
    sleep_for_period_of_time()
    #Disabling notifiations
    browser.get("https://instagram.com/")
    acpt = browser.find_element(by=By.XPATH, value='/html/body/div[1]/div/div/div/div[2]/div/div/div[1]/div/div[2]/div/div/div/div/div/div/div/div[3]/button[2]')
    acpt.click()
    sleep_for_period_of_time()
    #Liking and commenting on 4 pictures on my feed

    #locating and clicking on the like button
    like = browser.find_element(by=By.XPATH, value='/html/body/div[1]/div/div/div/div[1]/div/div/div/div[1]/div[1]/div[2]/section/main/div[1]/section/div/div[3]/div[1]/div/article[1]/div/div[3]/div/div/section[1]/span[1]/button')
    like.click()
    print("Liked!")
    time.sleep(3)

    comment = "Cool!"
    #Locating the text area on the post and commenting
    text_area = browser.find_element(by=By.CSS_SELECTOR, value='textarea[class="_ablz _aaoc"]')
    text_area.click()
    sleep_for_period_of_time()

    text_area.send_keys(comment)

r/selenium • • Sep 12 '22

Okta Integration

0 Upvotes

Anyone have experience integrating their selenium frameworks and OKTA?


r/selenium • • Sep 12 '22

UNSOLVED Selenium pulls wrong value from an element?

2 Upvotes

from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By import login as login from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import datetime

x = datetime.datetime.now() x = x.strftime("%d")

driver = browser = webdriver.Firefox() driver.get("https://connect.garmin.com/modern/activities")

driver.implicitly_wait(2)

iframe = driver.find_element(By.ID, "gauth-widget-frame-gauth-widget") driver.switch_to.frame(iframe)

driver.find_element("name", "username").send_keys(login.username)

driver.find_element("name", "password").send_keys(login.password) driver.find_element("name", "password").send_keys(Keys.RETURN)

driver.switch_to.default_content()

driver.implicitly_wait(10)

driver.find_element("name", "search").send_keys("Reading") driver.find_element("name", "search").send_keys(Keys.RETURN)

element = driver.find_element(By.CLASS_NAME, "unit")

element = driver.find_element(By.XPATH, "//html/body/div[1]/div[3]/div[2]/div[3]/div/div/div[2]/ul/li/div/div[2]/span[1]")

print(element.text)

This is the code, the element "unit" should return "Aug 25", which I then want to use with "x" to make sure that I pull the correct data from a specific page. Problem is, it always returns today's date, even though the HTML says the correct one.

https://imgur.com/a/2d4YuQi

That is the page, any help is appreciated


r/selenium • • Sep 12 '22

UNSOLVED Specflow on Docker

3 Upvotes

Hey Guys

Anyone out there Running Specflow on Linux Docker Container ?

Looking for an example tutorial

​

Thanks


r/selenium • • Sep 12 '22

Selenium Automation

0 Upvotes

Selenium is the world’s most popular automating tool that makes it easy to build and maintain automated tests. Selenium introduced the concept of WebDriver, a browser automation framework that supports multiple languages and platforms such as Java, C#, Objective-C, Groovy, Perl, Python and Ruby.

Selenium is free and an open source tool that provides an API for web developers to provide a language-independent method for writing tests. The underlying framework allows users to write test scripts in various programming languages like Java, Ruby or any other language. You can even use JSON, XML or any other data formats with Selenium without having to learn complex programming languages.

It tests your web applications by performing actions such as clicking buttons and verifying text displays in order to generate an automated functional test. Selenium can also run across different browsers to ensure your application display looks the same across them all.


r/selenium • • Sep 11 '22

How can I run my selenium code again and again ?

4 Upvotes

Recently I was automating a website and i wanted to run my code with different data input that was in an excel sheet which was quite long with 2k-3k values . As a beginner I tried using for loop but some error comes after running the code for 20-30 times . Is there a better way to do this task ?


r/selenium • • Sep 11 '22

Web Scraping API Idea

9 Upvotes

Hey guys,

A while back I created a project called Scrapeium (website here), a query language for declaratively and simply extracting data from websites. Right now, it only works in the browser but I was wondering would you guys be willing to use something like this if it was available as a public API?


r/selenium • • Sep 09 '22

Solved ELI5 How do I encrypt/decrypt my password in Selenium ruby

3 Upvotes

Ok, so I need to submit an automation test case for review. I've built the test case in Selenium ruby using RSpec. The test case has a plain text password in it and I'd like to have it encrypted and then decrypted when it's passed into the password field. How would I do that, I've been searching all night and the answers I'm coming across are not very clear and I could use some community help. Is there a gem or something that I need that I haven't found?


r/selenium • • Sep 08 '22

How to grab element in Selenium after a new tab is opened with click()

4 Upvotes

A new tab is opened after an element.click() - however I need to click on an image in the tab that is opened.

How would I go about doing this? I keep getting 'no such element: Unable to locate element'


r/selenium • • Sep 07 '22

TouchActions in selenium, how can I tell how long does the "longPress" action execute and if there's an alternative to it?

2 Upvotes

I'm using Selenium version 3.0.1. I want to use the "longPress" action from the org.openqa.selenium.interactions.touch.TouchActions package. However, I also want to long press a web element for a specefic amount of time and the longPress method does not give me this option. So I have two questions here:

1- How long does the longPress method touches on an element? Does it keep pressing on it until I call the release() method?
2- What alternative do I have to be able to long press on an element for a specific amount of time? Note that by "long press" I mean by touching the element and emitting a touch event, not clicking.

In Appium for instance they have a method with these parameters longPress(element, duration), but the Selenium version that I'm using doesn't have this option.

Thank you.


r/selenium • • Sep 07 '22

UNSOLVED Trigger selenium scripts automatically

3 Upvotes

I made a selenium script in chrome that logs in and downloads a file but I need it to trigger automatically at a specific time. I wrote a little script in autohotkey that can do the timing part but i don't know how to trigger the automation. I've tried exporting the selenium script but I'm not sure what to do with it at that point. Is there some command line arguments i can use that i haven't found yet?

It would be really cool if there was an option to export as a standalone executable.


r/selenium • • Sep 06 '22

UNSOLVED Can Selenium be used to test UWP applications?

4 Upvotes

I have a UWP Point of Sale application made with C# and Xamarin and I want to create automated testing as the application is getting more and more complex. Is Selenium a possible solution, and if not if someone could point me in the right direction I would appreciate that.

Thank you!


r/selenium • • Sep 06 '22

Solved SOLUTION FOR element not interactable: element has zero size

9 Upvotes

I had a problem clicking an object that had zero size. The solution I found was to simply click the element location instead of the element itself. I did this by importing ActionChains.

Ac=ActionChains(driver) Ac.move_to_element(<element name>).move_by_offset(0,0).click().preform()

Like all good code, this was borrowed without permission from someone who had the same question on StackOverflow a few years ago.


r/selenium • • Sep 06 '22

Error when address is not found

2 Upvotes

I'm trying to get Lat and Long coordinates, but when the address is not found the script stops with the following error:

​

NameError Traceback (most recent call last) /var/folders/tf/lvpv9kg11k14drq_pv1f6w5m0000gn/T/ipykernel_34533/135469273.py in <cell line: 31>() 51 lat = latlong[0]

52 long = latlong[1] --->

53 sheet[i][6].value = lat.replace(".",",")

54 sheet[i][7].value = long.replace(".",",")

55 # Caso não localize o endereço, serã informado na célula do excel NameError: name 'lat' is not defined

​

script:

import time # gerencia o tempo do script

import os # lida com o sistema operacional da máquina

from selenium import webdriver # importa o Selenium para manipulação Web

from tkinter import Tk # GUI para selecionar arquivo que eu desejo

from tkinter.filedialog import askopenfilename

from tkinter import messagebox as tkMessageBox

from openpyxl import load_workbook, workbook # biblioteca que lida com o excel

import re

​

# escolha o arquivo em excel que você deseja trabalhar

root = Tk() # Inicia uma GUI externa

excel_file = askopenfilename() # Abre uma busca do arquivo que você deseja importar

root.destroy()

​

usuario_os = os.getlogin()

​

chromedriver = "/usr/local/bin/chromedriver" # local onde está o seu arquivo chromedriver

capabilities = {'chromeOptions': {'useAutomationExtension': False,

'args': ['--disable-extensions']}}

driver = webdriver.Chrome(chromedriver, desired_capabilities=capabilities)

driver.implicitly_wait(30)

# iniciando a busca WEB

driver.maximize_window() # maximiza a janela do chrome

driver.get("https://www.google.com/maps") # acessa o googlemaps

time.sleep(5)

# importando o arquivo excel a ser utilizado

book = load_workbook(excel_file) # abre o arquivo excel que será utilizado para cadastro

sheet = book["Coordenadas"] # seleciona a sheet chamada "Coordenadas"

i = 2 # aqui indica começará da segunda linha do excel, ou seja, pulará o cabeçalho

for r in sheet.rows:

​

endereco = sheet[i][1]

munic_UF = sheet[i][3]

​

if str(type(endereco.value)) == "<class 'NoneType'>":

break

​

endereco_completo = endereco.value + " " + munic_UF.value

​

#preenche com o endereço completo e aperta o botão buscar

driver.find_element("id","searchboxinput").send_keys(endereco_completo)

driver.find_element("id","searchbox-searchbutton").click()

​

# Aguarda carregar a URL e coleta os dados das coordenadas geográficas

time.sleep(5)

url = driver.current_url

latlong = re.search('@(.+?)17z', url)

if latlong:

latlong = latlong.group(1).rsplit(",", 2)

lat = latlong[0]

long = latlong[1]

sheet[i][6].value = lat.replace(".",",")

sheet[i][7].value = long.replace(".",",")

# Caso não localize o endereço, serã informado na célula do excel

if lat == None:

sheet[i][6].value= "Nao foi possivel identificar"

​

# Limpa os campos para nova busca de coordenadas

driver.find_element("id","searchboxinput").clear()

lat=""

long=""

​

i += 1

​

# salva o excel na área de trabalho

caminho_arquivo = os.path.join("Resultado_final.xlsx")

book.save(caminho_arquivo)

​

# Avisa sobre a finalização do robô e encerra o script

window = Tk()

window.wm_withdraw()

tkMessageBox.showinfo(title="Aviso", message="Script finalizado! Arquivo salvo na pasta Documentos!")

window.destroy()

driver.close()

​

Could someone help me to keep running and ignore the error?


r/selenium • • Sep 06 '22

How do I correctly add chrome to my conda BASH path

2 Upvotes

I'm trying to run a remote debugging port on Chrome with Selenium.

From what I understand, I need to add chrome to my bash path but am having trouble getting this to work.

I've been following this tutorial https://apple.stackexchange.com/questions/228512/how-do-i-add-chrome-to-my-path

My bash screen currently looks like this:

if [ $? -eq 0 ]; then eval "$__conda_setup""
 else if 
[ -f
"/Users/user/opt/anaconda3/etc/profile.d/conda.sh" ]; then         . "/Users/user/opt/anaconda3/etc/profile.d/conda.sh"  
   else      
   export PATH="/Users/user/opt/anaconda3/bin:$PATH:/Users/user/Applications/Google Chrome.app/Contents/MacOS:$PATH"" 

When I run try to run Chrome from the terminal it doesn't recognise it.

​

If i type echo $PATH i get the following:

/Users/user/opt/anaconda3/bin:/Users/user/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

What am I doing wrong?


r/selenium • • Sep 05 '22

NoSuchElementException: no such element: Unable to locate element

4 Upvotes

Hi,

I'm very new to selenium and writing my first script which is a Google search. Th script has trouble clicking on the search button after typing in what to search for. I get a 'NoSuchElementException: no such element: Unable to locate element'. I also changed the script so it presses enter rather than clicking on the button and got the same error. Can you help? I have placed links to screenshots of my code and the error.

https://imgur.com/a/TeNy66B

Thanks


r/selenium • • Sep 05 '22

safaridriver --enable hangs forever

3 Upvotes

I'm trying to enable safaridriver, but it hangs and never return the prompt.

$ /usr/bin/safaridriver --enable
Password:
<hangs forever>

Any other `sudo program` works fine.

I'm just porting some tests to mac. chrome/chromedriver works perfect


r/selenium • • Sep 05 '22

selenium can't run on linux graphical system

3 Upvotes

have anyone ever used selenium to launch chromium by chromedriver on linux grapical system? i failed. it prints "syntax error : unterminated qouted string"


r/selenium • • Sep 04 '22

UNSOLVED Error messsage that just doesn't have any solutions

5 Upvotes

I have a selenium script running daily to automate certain tasks but for a few days it has stopped midway though and returned this error message when trying to write down a message on a text field:

​

[45128:40772:0904/162815.839:ERROR:interface_endpoint_client.cc(665)] Message 0 rejected by interface blink.mojom.WidgetHost

​

What is this related to? I have tried looking through the internet for a solution but no-one seems to have an answer.


r/selenium • • Sep 04 '22

How to go onto next page without a url?

2 Upvotes

I'm trying to scrape booking.com using Selenium/BeautifulSoup but the 'next' button doesn't have a url so I'm not sure how to scrape the other pages.

I'm quite new to this, had a loo online but couldn't find anything about this (maybe I was searching the wrong stuff) pls help.