Build Frontend ETL Monitor Application

[shreyasgm]

Develop a simple, standalone Streamlit / Next.js application to display the ETL publication status fetched from the new backend API endpoint.

Context:
This provides the user interface for monitoring the ETL pipeline's progress. This is a separate application from the main query frontend.

Tasks:

1. Create a new directory for this frontend, e.g., etl_monitor_frontend/.
2. Inside this directory, create a Streamlit application file (e.g., app.py).
3. Add necessary dependencies (streamlit, requests) - potentially in a etl_monitor_frontend/pyproject.toml if managing dependencies separately or add to the main pyproject.toml under optional dependencies.
4. Implement the Streamlit app to:
   • Call the backend API endpoint created in Issue Basic ETL pipeline starter code #6 (use the requests library). The API URL should be configurable via environment variables (etl_monitor_frontend/.env).
   • Provide UI elements (e.g., dropdowns, date pickers) for filtering/sorting options corresponding to the API parameters.
   • Display the fetched data in a clear tabular format (e.g., using st.dataframe).
   • Include a refresh button.
   • Handle API errors gracefully (e.g., display an error message if the backend is unreachable).
5. Create a Dockerfile for this new frontend service (etl_monitor_frontend/Dockerfile).
6. Create an .env.example file for environment variables (like the backend API URL).

Acceptance Criteria:

• A Streamlit application exists in etl_monitor_frontend/.
• The application successfully calls the backend API /etl/status/.
• Publication data is displayed in a table.
• Users can filter/sort the data via UI controls.
• The application has a Dockerfile and .env.example.