인프런 커뮤니티 질문&답변
작성한 질문수
docker-compose up 이후 웹서버가 뜨지 않습니다
해결된 질문
작성
·
708
0
안녕하세요
도커&에어플로우 설치 부분 강사님 강의보며 실습하던 중에 크롬창에서 웹서버 접속이 되지 않아 문의드립니다.
환경은 mac os 입니다.
sudo docker-compose up airflow-init
sudo docker-compose up
명령어 사용해서 진행했습니다.
로그 중에 permission denied가 나는 부분이 있는데 이 부분 때문일까요?
airflow-init-1 | chown: changing ownership of '/sources/logs': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/dags': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/plugins': Permission denied
로그 전문 공유드립니다.
(airflow-dockercompose) jeremy 😹 ~/dev/airflow-dockercompose sudo docker-compose up airflow-init
Password:
[+] Running 44/3
✔ postgres 13 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 72.8s
✔ redis 6 layers [⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 84.6s
✔ airflow-init 22 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿] 0B/0B Pulled 68.5s
[+] Running 3/3
✔ Container airflow-dockercompose-redis-1 Created 0.3s
✔ Container airflow-dockercompose-postgres-1 Created 0.3s
✔ Container airflow-dockercompose-airflow-init-1 Created 0.0s
Attaching to airflow-init-1
airflow-init-1 | chown: changing ownership of '/sources/logs': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/dags': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/plugins': Permission denied
airflow-init-1 | The container is run as root user. For security, consider using a regular user account.
airflow-init-1 |
airflow-init-1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init-1 | Performing upgrade to the metadata database postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init-1 | [2024-02-13T12:36:11.330+0000] {migration.py:216} INFO - Context impl PostgresqlImpl.
airflow-init-1 | [2024-02-13T12:36:11.332+0000] {migration.py:219} INFO - Will assume transactional DDL.
airflow-init-1 | [2024-02-13T12:36:11.344+0000] {db.py:1616} INFO - Creating tables
airflow-init-1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init-1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-init-1 | Database migrating done!
airflow-init-1 | /home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:336 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
airflow-init-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-init-1 | airflow already exist in the db
airflow-init-1 | 2.8.1
airflow-init-1 exited with code 0
(airflow-dockercompose) jeremy 😹 ~/dev/airflow-dockercompose sudo docker-compose up
[+] Running 7/7
✔ Container airflow-dockercompose-postgres-1 Running 0.0s
✔ Container airflow-dockercompose-redis-1 Running 0.0s
✔ Container airflow-dockercompose-airflow-init-1 Created 0.0s
✔ Container airflow-dockercompose-airflow-worker-1 Created 0.1s
✔ Container airflow-dockercompose-airflow-scheduler-1 Created 0.1s
✔ Container airflow-dockercompose-airflow-triggerer-1 Created 0.1s
✔ Container airflow-dockercompose-airflow-webserver-1 Created 0.1s
Attaching to airflow-init-1, airflow-scheduler-1, airflow-triggerer-1, airflow-webserver-1, airflow-worker-1, postgres-1, redis-1
airflow-init-1 | chown: changing ownership of '/sources/logs': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/dags': Permission denied
airflow-init-1 | chown: changing ownership of '/sources/plugins': Permission denied
airflow-init-1 | The container is run as root user. For security, consider using a regular user account.
airflow-init-1 |
airflow-init-1 | DB: postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init-1 | Performing upgrade to the metadata database postgresql+psycopg2://airflow:***@postgres/airflow
airflow-init-1 | [2024-02-13T12:37:00.495+0000] {migration.py:216} INFO - Context impl PostgresqlImpl.
airflow-init-1 | [2024-02-13T12:37:00.496+0000] {migration.py:219} INFO - Will assume transactional DDL.
airflow-init-1 | [2024-02-13T12:37:00.509+0000] {db.py:1616} INFO - Creating tables
airflow-init-1 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
airflow-init-1 | INFO [alembic.runtime.migration] Will assume transactional DDL.
airflow-init-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-init-1 | Database migrating done!
airflow-init-1 | /home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:336 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
airflow-init-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-init-1 | airflow already exist in the db
airflow-init-1 | 2.8.1
airflow-init-1 exited with code 0
airflow-triggerer-1 |
airflow-worker-1 |
airflow-webserver-1 |
airflow-scheduler-1 |
airflow-webserver-1 | [2024-02-13T12:37:28.080+0000] {configuration.py:2065} INFO - Creating new FAB webserver config file in: /opt/airflow/webserver_config.py
airflow-triggerer-1 | ____________ _____________
airflow-triggerer-1 | ____ |__( )_________ __/__ /________ __
airflow-triggerer-1 | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
airflow-triggerer-1 | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
airflow-triggerer-1 | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
답변 2
0
신윤철
질문자
airflow-scheduler-1 | [2024-02-13T12:37:38.821+0000] {task_context_logger.py:63} INFO - Task context logging is enabled
airflow-scheduler-1 | [2024-02-13T12:37:38.822+0000] {executor_loader.py:115} INFO - Loaded executor: CeleryExecutor
airflow-scheduler-1 | [2024-02-13T12:37:39.321+0000] {scheduler_job_runner.py:808} INFO - Starting the scheduler
airflow-scheduler-1 | [2024-02-13T12:37:39.337+0000] {scheduler_job_runner.py:815} INFO - Processing each file at most -1 times
airflow-scheduler-1 | [2024-02-13T12:37:39.343+0000] {manager.py:169} INFO - Launched DagFileProcessorManager with pid: 28
airflow-scheduler-1 | [2024-02-13T12:37:39.345+0000] {scheduler_job_runner.py:1619} INFO - Adopting or resetting orphaned tasks for active dag runs
airflow-scheduler-1 | [2024-02-13T12:37:39.361+0000] {settings.py:60} INFO - Configured default timezone UTC
airflow-worker-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-worker-1 |
airflow-worker-1 | -------------- celery@4f4dfc593d9d v5.3.6 (emerald-rush)
airflow-worker-1 | --- ***** -----
airflow-worker-1 | -- ******* ---- Linux-5.15.96-0-virt-x86_64-with-glibc2.34 2024-02-13 12:37:40
airflow-worker-1 | - *** --- * ---
airflow-worker-1 | - ** ---------- [config]
airflow-worker-1 | - ** ---------- .> app: airflow.providers.celery.executors.celery_executor:0x7ff49d2ff640
airflow-worker-1 | - ** ---------- .> transport: redis://redis:6379/0
airflow-worker-1 | - ** ---------- .> results: postgresql://airflow:**@postgres/airflow
airflow-worker-1 | - *** --- * --- .> concurrency: 16 (prefork)
airflow-worker-1 | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
airflow-worker-1 | --- ***** -----
airflow-worker-1 | -------------- [queues]
airflow-worker-1 | .> default exchange=default(direct) key=default
airflow-worker-1 |
airflow-worker-1 |
airflow-worker-1 | [tasks]
airflow-worker-1 | . airflow.providers.celery.executors.celery_executor_utils.execute_command
airflow-worker-1 |
airflow-triggerer-1 | [2024-02-13T12:37:46.699+0000] {triggerer_job_runner.py:576} INFO - Triggerer's async thread was blocked for 0.20 seconds, likely by a badly-written trigger. Set PYTHONASYNCIODEBUG=1 to get more information on overrunning coroutines.
airflow-worker-1 | [2024-02-13 12:37:52,175: WARNING/MainProcess] /home/airflow/.local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
airflow-worker-1 | whether broker connection retries are made during startup in Celery 6.0 and above.
airflow-worker-1 | If you wish to retain the existing behavior for retrying connections on startup,
airflow-worker-1 | you should set broker_connection_retry_on_startup to True.
airflow-worker-1 | warnings.warn(
airflow-worker-1 |
airflow-worker-1 | [2024-02-13 12:37:52,258: INFO/MainProcess] Connected to redis://redis:6379/0
airflow-worker-1 | [2024-02-13 12:37:52,259: WARNING/MainProcess] /home/airflow/.local/lib/python3.8/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
airflow-worker-1 | whether broker connection retries are made during startup in Celery 6.0 and above.
airflow-worker-1 | If you wish to retain the existing behavior for retrying connections on startup,
airflow-worker-1 | you should set broker_connection_retry_on_startup to True.
airflow-worker-1 | warnings.warn(
airflow-worker-1 |
airflow-webserver-1 | [2024-02-13T12:37:52.263+0000] {providers_manager.py:280} INFO - Optional provider feature disabled when importing 'airflow.providers.google.leveldb.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package
airflow-worker-1 | [2024-02-13 12:37:52,268: INFO/MainProcess] mingle: searching for neighbors
airflow-worker-1 | [2024-02-13 12:37:53,308: INFO/MainProcess] mingle: all alone
airflow-worker-1 | [2024-02-13 12:37:53,441: INFO/MainProcess] celery@4f4dfc593d9d ready.
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [16] [INFO] Listening at: http://0.0.0.0:8080 (16)
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [16] [INFO] Using worker: sync
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [27] [INFO] Booting worker with pid: 27
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [28] [INFO] Booting worker with pid: 28
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [29] [INFO] Booting worker with pid: 29
airflow-webserver-1 | [2024-02-13 12:37:57 +0000] [30] [INFO] Booting worker with pid: 30
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:38:11 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-triggerer-1 | [2024-02-13T12:38:30.979+0000] {triggerer_job_runner.py:481} INFO - 0 triggers currently running
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:38:41 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:39:11 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-triggerer-1 | [2024-02-13T12:39:31.176+0000] {triggerer_job_runner.py:481} INFO - 0 triggers currently running
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:39:41 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:40:12 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-triggerer-1 | [2024-02-13T12:40:31.368+0000] {triggerer_job_runner.py:481} INFO - 0 triggers currently running
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:40:42 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:41:12 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-triggerer-1 | [2024-02-13T12:41:31.550+0000] {triggerer_job_runner.py:481} INFO - 0 triggers currently running
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:41:42 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-webserver-1 | 127.0.0.1 - - [13/Feb/2024:12:42:12 +0000] "GET /health HTTP/1.1" 200 318 "-" "curl/7.88.1"
airflow-triggerer-1 | [2024-02-13T12:42:31.748+0000] {triggerer_job_runner.py:481} INFO - 0 triggers currently running0
신윤철
질문자
10000자 제한이 있어 답글로 나머지 로그 공유드려요
airflow-triggerer-1 | [2024-02-13 12:37:29 +0000] [15] [INFO] Starting gunicorn 21.2.0
airflow-triggerer-1 | [2024-02-13 12:37:29 +0000] [15] [INFO] Listening at: http://[::]:8794 (15)
airflow-triggerer-1 | [2024-02-13 12:37:29 +0000] [15] [INFO] Using worker: sync
airflow-triggerer-1 | [2024-02-13 12:37:29 +0000] [16] [INFO] Booting worker with pid: 16
airflow-triggerer-1 | [2024-02-13 12:37:29 +0000] [17] [INFO] Booting worker with pid: 17
airflow-triggerer-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-triggerer-1 | [2024-02-13T12:37:30.689+0000] {triggerer_job_runner.py:174} INFO - Setting up TriggererHandlerWrapper with handler <FileTaskHandler (NOTSET)>
airflow-triggerer-1 | [2024-02-13T12:37:30.700+0000] {triggerer_job_runner.py:230} INFO - Setting up logging queue listener with handlers [<RedirectStdHandler <stdout> (NOTSET)>, <TriggererHandlerWrapper (NOTSET)>]
airflow-triggerer-1 | [2024-02-13T12:37:30.726+0000] {triggerer_job_runner.py:331} INFO - Starting the triggerer
airflow-webserver-1 | /home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:336 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
airflow-worker-1 | BACKEND=redis
airflow-worker-1 | DB_HOST=redis
airflow-worker-1 | DB_PORT=6379
airflow-worker-1 |
airflow-scheduler-1 | BACKEND=redis
airflow-scheduler-1 | DB_HOST=redis
airflow-scheduler-1 | DB_PORT=6379
airflow-scheduler-1 |
airflow-webserver-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-worker-1 | [2024-02-13T12:37:36.823+0000] {configuration.py:2065} INFO - Creating new FAB webserver config file in: /opt/airflow/webserver_config.py
airflow-webserver-1 | [2024-02-13T12:37:37.571+0000] {options.py:83} WARNING - The swagger_ui directory could not be found.
airflow-webserver-1 | Please install connexion with extra install: pip install connexion[swagger-ui]
airflow-webserver-1 | or provide the path to your local installation by passing swagger_path=<your path>
airflow-webserver-1 |
airflow-webserver-1 | [2024-02-13T12:37:37.577+0000] {options.py:83} WARNING - The swagger_ui directory could not be found.
airflow-webserver-1 | Please install connexion with extra install: pip install connexion[swagger-ui]
airflow-webserver-1 | or provide the path to your local installation by passing swagger_path=<your path>
airflow-webserver-1 |
airflow-webserver-1 | [2024-02-13 12:37:37 +0000] [16] [INFO] Starting gunicorn 21.2.0
airflow-scheduler-1 | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
airflow-worker-1 | [2024-02-13 12:37:38 +0000] [26] [INFO] Starting gunicorn 21.2.0
airflow-worker-1 | [2024-02-13 12:37:38 +0000] [26] [INFO] Listening at: http://[::]:8793 (26)
airflow-worker-1 | [2024-02-13 12:37:38 +0000] [26] [INFO] Using worker: sync
airflow-worker-1 | [2024-02-13 12:37:38 +0000] [27] [INFO] Booting worker with pid: 27
airflow-worker-1 | [2024-02-13 12:37:38 +0000] [28] [INFO] Booting worker with pid: 28
airflow-scheduler-1 | ____________ _____________
airflow-scheduler-1 | ____ |__( )_________ __/__ /________ __
airflow-scheduler-1 | ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
airflow-scheduler-1 | ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
airflow-scheduler-1 | _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
신윤철
질문자
현진님 답글 달아주셔서 감사드립니다.
저도 방금 에어플로우를 띄웠는데 말씀주신 방법 말고 다른 방법으로 띄워 공유드려요
홈브루로 설치했던 도커를 지우고 colima 로 설치해서 해결(?) 했습니다.
완벽하게 이해하고 띄운것은 아니라서 현진님께서 말씀주신 내용 바탕으로 강의 다 듣고 테스트 해보겠습니다.
감사합니다 🙂
제가 참고했던 링크 함께 공유드립니다.
콜리마 설치
https://how.wtf/how-to-use-docker-without-docker-desktop-on-macos.html
도커컴포즈
https://smallsharpsoftwaretools.com/tutorials/use-colima-to-run-docker-containers-on-macos/
에어플로우 문서
https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html
안녕하세요 신윤철님!
아래 3개 디렉토리의 권한을 점검해봐야할 것 같습니다.
$HOME/source/dags
$HOME/source/logs
$HOME/source/plugins
ls -al 명령으로 위 디렉토리 owner를 보시겠어요?
root owner에 root group 권한으로 만들어졌을수도 있습니다.
그럼 owner 를 현재 로그인된 유저의 권한으로 변경해주셔야 해요.
한번 확인해보시겠어요?