WebDriverWait
对象来等待元素加载完成后再获取内容。示例代码:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# 确定WebDrive已定义并启用
driver.get('http://example.com')
wait = WebDriverWait(driver, 10) # 最多等待10秒
# 等待元素加载完成后获取内容
element = wait.until(EC.presence_of_element_located((By.ID, 'id_of_element')))
content = element.text
示例代码:
# 判断元素是否存在
if driver.find_elements_by_css_selector('div#id_of_element'):
element = driver.find_element_by_css_selector('div#id_of_element')
content = element.text
示例代码:
# 使用requests库获取网页内容
import requests
from bs4 import BeautifulSoup
response = requests.get('http://example.com')
soup = BeautifulSoup(response.text, 'html.parser')
content = soup.select_one('div#id_of_element').text
通过以上方法,我们可以很好地解决Beautiful Soup和Selenium无法获取内容的问题。