EmailOperator 강습 중에 실행 오류 관련 문의 드립니다... - 인프런

Airflow 마스터 클래스

Email Operator로 메일 전송하기

EmailOperator 강습 중에 실행 오류 관련 문의 드립니다.

작성

157

수정됨

강사님

*** !!!! Please make sure that all your Airflow components (e.g. schedulers, webservers, workers and triggerer) have the same 'secret_key' configured in 'webserver' section and time is synchronized on all your machines (for example with ntpd) See more at https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key *** Could not read served logs: 403 Client Error: FORBIDDEN for url: http://e1efbc97ae25:8793/log/dag_id=dags_email_operator/run_id=manual__2024-08-24T06:22:20.118495+00:00/task_id=send_email_task/attempt=1.log [2024-08-24, 06:22:25 UTC] {local_task_job_runner.py:123} ▶ Pre task execution logs [2024-08-24, 06:22:25 UTC] {warnings.py:112} WARNING - /home/***/.local/lib/python3.12/site-packages/***/utils/email.py:155: RemovedInAirflow3Warning: Fetching SMTP credentials from configuration variables will be deprecated in a future release. Please set credentials using a connection instead. send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)

[2024-08-24, 06:27:49 UTC] {taskinstance.py:3301} ERROR - Task failed with exception Traceback (most recent call last): File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 767, in executetask result = executecallable(context=context, **execute_callable_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 733, in executecallable return ExecutionCallableRunner( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run return self.func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 406, in wrapper return func(self, args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/email.py", line 79, in execute send_email( File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 80, in send_email return backend( ^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 155, in send_email_smtp send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun) File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 280, in send_mime_email smtp_conn.starttls() File "/usr/local/lib/python3.12/smtplib.py", line 779, in starttls self.sock = context.wrap_socket(self.sock, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1042, in create self.dohandshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() TimeoutError: _ssl.c:983: The handshake operation timed out

위 문구와 함께 현재 smtp.gmail.com에 접속이 안되는데.. (worker에서도 접속이 안됨)

혹시 다른 설정이 필요할게 있을까요?

python 데이터-엔지니어링 airflow

답변 3

walnam777

질문자

강사님

docker worker node안에 들어가서 테스트 했었습니다.

강사님 말씀대로 tls에서 막히네요..ㅠ

그런데 로컬 cmd에서 진행시에는 문제 없이 됩니다.

이건 제 WSL환경 문제 일까요?

walnam777

질문자

강사님

Post Task Execution log부분입니다.

[2024-08-24, 05:40:08 UTC] {taskinstance.py:340} ▼ Post task execution logs [2024-08-24, 05:40:08 UTC] {standard_task_runner.py:124} ERROR - Failed to execute job 29 for task send_email_task (_ssl.c:983: The handshake operation timed out; 162) Traceback (most recent call last): File "/home/airflow/.local/lib/python3.12/site-packages/airflow/task/task_runner/standard_task_runner.py", line 117, in startby_fork ret = args.func(args, dag=self.dag) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py", line 49, in command return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line 115, in wrapper return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/task_command.py", line 483, in task_run task_return_code = runtask_by_selected_method(args, dag, ti) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/taskcommand.py", line 256, in runtask_by_selected_method return runraw_task(args, ti) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/task_command.py", line 341, in runraw_task return ti._run_raw_task( ^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py", line 97, in wrapper return func(*args, session=session, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 2995, in runraw_task return runraw_task( ^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 273, in runraw_task TaskInstance._execute_task_with_callbacks( File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3149, in executetask_with_callbacks result = self._execute_task(context, task_orig) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3173, in executetask return executetask(self, context, task_orig) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 767, in executetask result = executecallable(context=context, execute_callable_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 733, in executecallable return ExecutionCallableRunner( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run return self.func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 406, in wrapper return func(self, args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/email.py", line 79, in execute send_email( File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 80, in send_email return backend( ^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 155, in send_email_smtp send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun) File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/email.py", line 280, in send_mime_email smtp_conn.starttls() File "/usr/local/lib/python3.12/smtplib.py", line 779, in starttls self.sock = context.wrap_socket(self.sock, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/ssl.py", line 1042, in create self.dohandshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() TimeoutError: _ssl.c:983: The handshake operation timed out

김현진

지식공유자

음 별다른 로그는 없네요.

혹시 워커에서도 smtp.gmail.com 접속이 안되는건 어떻게 확인하셨어요?

워커 컨테이너 접속해서 아래처럼 파이썬 들어간 다음 아래 명령 해보시겠어요?

% python 
>>> import smtplib
>>> smtp = smtplib.SMTP('smtp.gmail.com', 587)
>>> smtp.starttls()
(220, b'2.0.0 Ready to start TLS')

로그상에서는 starttls() 부분에서 막힌것 같긴 합니다.

이 부분에서 안되면 로컬 컴퓨터에서도 동일하게 해서 잘 되는지 확인 부탁드립니다.

김현진

지식공유자

walnam777님

증상으로 보아 네트워크나 DNS에는 문제 없고 SSL 연결하는 부분에서 막히고 있네요.

혹시 실습하시는 환경이 어디인가요?

댁이신가요? 회사인가요?

김현진

지식공유자

안녕하세요 walnm777님

혹시 로그 중에 Post task Execution 부분에 로그는 없나요? 해당 내용도 업로드 해주시겠어요?

인프런 커뮤니티 질문&답변

EmailOperator 강습 중에 실행 오류 관련 문의 드립니다.