인프런 커뮤니티 질문&답변

안녕하세요

리뷰 크롤링 하는 과정에서 다른 페이지에 적용 학습해보다가 에러코드가 떠서 질문드립니다.

name                = ['언더아머 CGI 다운']
ns_address          = 'https://search.shopping.naver.com/search/all?query=cgi%20%EB%8B%A4%EC%9A%B4&frm=NVSHATC&prevQuery=%EB%89%B4%EB%B0%9C%EB%9E%80%EC%8A%A4%EB%B0%94%EB%9E%8C%EB%A7%89%EC%9D%B4'
shoppingmall_review = "/html/body/div/div/div[2]/div[2]/div[2]/div[3]/div[1]/ul/li[3]/a"
category_total      = "/html/body/div/div/div[2]/div[2]/div[2]/div[3]/div[7]/div[2]/div[2]/ul/li[1]/a"

순서대로 위와 같이 적용 완료하였는데요, 그 다음에 소스코드 주신 부분인 아래 부분을 적용하니까 다음과 같은 에러가 뜹니다. 확인 부탁드려도 될까요?

header = {'User-Agent': ''} driver.implicitly_wait(3) driver.get(ns_address) req = requests.get(ns_address,verify=False) html = req.text soup = BeautifulSoup(html, "html.parser") sleep(2) element=driver.find_element_by_xpath(shoppingmall_review) driver.execute_script("arguments[0].click();", element) sleep(2)'

에러 -

/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)

---------------------------------------------------------------------------

NoSuchElementException                    Traceback (most recent call last)

<ipython-input-30-8a5c5adbf17a> in <module>
      6 soup = BeautifulSoup(html, "html.parser")
      7 sleep(2)
----> 8 element=driver.find_element_by_xpath(shoppingmall_review)
      9 driver.execute_script("arguments[0].click();", element)
     10 sleep(2)

3 frames

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py in find_element_by_xpath(self, xpath)
    392             element = driver.find_element_by_xpath('//div/td[1]')
    393         """
--> 394         return self.find_element(by=By.XPATH, value=xpath)
    395 
    396     def find_elements_by_xpath(self, xpath):

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py in find_element(self, by, value)
    976         return self.execute(Command.FIND_ELEMENT, {
    977             'using': by,
--> 978             'value': value})['value']
    979 
    980     def find_elements(self, by=By.ID, value=None):

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
    319         response = self.command_executor.execute(driver_command, params)
    320         if response:
--> 321             self.error_handler.check_response(response)
    322             response['value'] = self._unwrap_value(
    323                 response.get('value', None))

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    240                 alert_text = value['alert'].get('text')
    241             raise exception_class(message, screen, stacktrace, alert_text)
--> 242         raise exception_class(message, screen, stacktrace)
    243 
    244     def _value_or_default(self, obj, key, default):

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div/div/div[2]/div[2]/div[2]/div[3]/div[1]/ul/li[3]/a"}
  (Session info: headless chrome=105.0.5195.102)

그리고 다음 url과 같이 나타낼 페이지가 그리 많지 않은 경우에도 같은 소스코드 양식을 사용해도 될까요?

https://search.shopping.naver.com/catalog/29274895216?query=cgi%20%EB%8B%A4%EC%9A%B4&NaPm=ct%3Dl8qn5dbs%7Cci%3Dcc97712ba6dec8be52ea670a2e607bb755d59f4f%7Ctr%3Dslsl%7Csn%3D95694%7Chk%3D97fd029ce7fc81750ad0a1d2110ad155b54fe09d

감사합니다.