diff --git a/.dockerignore b/.dockerignore index 70f3ebe..8267d8b 100644 --- a/.dockerignore +++ b/.dockerignore @@ -38,4 +38,5 @@ http-client.private.env.json resources/poster_cache deployment prowlarr-config/ -config/ \ No newline at end of file +config/ +mediafusion.local* \ No newline at end of file diff --git a/.gitignore b/.gitignore index 837cf2c..6077cbe 100644 --- a/.gitignore +++ b/.gitignore @@ -138,4 +138,5 @@ http-client.private.env.json resources/poster_cache prowlarr-config/ -config/ \ No newline at end of file +config/ +mediafusion.local* \ No newline at end of file diff --git a/README.md b/README.md index 7fec575..6cc5550 100644 --- a/README.md +++ b/README.md @@ -21,75 +21,13 @@ ## :rocket: Installation 1. Install Stremio from [here](https://www.stremio.com/downloads). -2. Navigate to [Media Fusion](https://882b9915d0fe-mediafusion.baby-beamup.club) and click on the 'Configure Add-on' button. +2. Navigate to [Media Fusion](https://mediafusion.fun) and click on the 'Configure Add-on' button. -## :hammer_and_wrench: Development - -### Prerequisites - -- **Python**: This project uses Python version 3.11. Ensure you have it installed. -- **MongoDB**: Set up a MongoDB server. You can use [MongoDB Atlas](https://www.mongodb.com/cloud/atlas) to create a free MongoDB cluster. -- **mkcert**: To set up local HTTPS, you'll need to use mkcert to generate SSL certificates. If not installed, get it from [here](https://github.com/FiloSottile/mkcert). - -### Setup - -1. **Pipenv**: Use [Pipenv](https://pipenv.pypa.io/en/latest/) for managing project dependencies. If you don't have it installed, you can install it with: - ```bash - pip install pipenv - ``` -2. **Clone**: Clone this repository. - ```bash - git clone https://github.com/mhdzumair/MediaFusion - ``` -3. **Install Dependencies**: Navigate to the MediaFusion directory and install dependencies with: - ```bash - pipenv install - ``` -4. **Environment Variables**: Create a `.env` file in the root directory with the following variables: - ```bash - MONGO_URI= - SECRET_KEY= - HOST_URL=https://127.0.0.1:8443 - ``` -5. **Local HTTPS Setup**: - - - Navigate to the MediaFusion directory. - - Generate local SSL certificates using mkcert: - - ```bash - mkcert -install - mkcert 127.0.0.1 - ``` - - This will generate two files: localhost.pem and localhost-key.pem. - -6. **Run Servers**: - - - To serve application over HTTPS on port 8443: - - ```bash - pipenv run uvicorn api.main:app --host 127.0.0.1 --port 8443 --ssl-keyfile 127.0.0.1-key.pem --ssl-certfile 127.0.0.1.pem - ``` - - - Since Stremio doesn't support localhost HTTPS servers to install add-on, also run an HTTP server on port 8000: - - ```bash - pipenv run uvicorn api.main:app --host 127.0.0.1 --port 8000 - ``` - -7. **For scraping instructions**: refer to the [scrapping README](/scrappers/README.md). +## :rocket: Local Add-on Deployment +check the [Local Deployment Guide](deployment/README.md) for detailed instructions. -## :heart: Acknowledgments - - - AnTuDu's Profile Picture - - - -Special thanks to [AnTuDu](https://github.com/AnTuDu) for providing the infrastructure support for deploying MediaFusion. - -### :sparkles: Contributors +## :sparkles: Contributors A special thank you to all our contributors! diff --git a/deployment/README.md b/deployment/README.md new file mode 100644 index 0000000..4aef2ef --- /dev/null +++ b/deployment/README.md @@ -0,0 +1,134 @@ + +# MediaFusion Local Deployment Guide 🚀 + +This guide provides instructions for deploying MediaFusion locally using Minikube, tailored for Windows, Linux, and macOS platforms. Follow these steps to set up MediaFusion on your local machine. + +## Clone the Repository + +```bash +git clone https://github.com/mhdzumair/MediaFusion +cd MediaFusion + +# Switch to the development branch +git checkout develop +``` + +## Prerequisites + +Before you begin, ensure the following tools are installed on your system: + +- **Minikube**: For local Kubernetes development. Install instructions are available in the [Minikube documentation](https://minikube.sigs.k8s.io/docs/start/). + +- **kubectl**: The Kubernetes command-line tool. Installation guidelines can be found in the [Kubernetes documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/). + +- **Python 3.11**: Required for mkcert & development. Follow the [Python documentation](https://www.python.org/downloads/) for installation instructions. + +## Setting Up Secrets 🗝️ + +MediaFusion requires certain secrets for operation. Use the following commands to create them: + +```bash +# Generate a random 32-character string for the SECRET_KEY +SECRET_KEY=$(openssl rand -hex 16) + +# Generate a random API key for Prowlarr +PROWLARR_API_KEY=$(openssl rand -hex 16) + +# If using Premiumize, fill in your OAuth client ID and secret. Otherwise, leave these empty. +PREMIUMIZE_OAUTH_CLIENT_ID="" +PREMIUMIZE_OAUTH_CLIENT_SECRET="" + +kubectl create secret generic mediafusion-secrets \ + --from-literal=SECRET_KEY=$SECRET_KEY \ + --from-literal=PROWLARR_API_KEY=$PROWLARR_API_KEY \ + --from-literal=PREMIUMIZE_OAUTH_CLIENT_ID=$PREMIUMIZE_OAUTH_CLIENT_ID \ + --from-literal=PREMIUMIZE_OAUTH_CLIENT_SECRET=$PREMIUMIZE_OAUTH_CLIENT_SECRET +``` + +### Updating Secrets + +To update existing secrets, use the following command: + +```bash +kubectl create secret generic mediafusion-secrets \ + --from-literal=SECRET_KEY=$SECRET_KEY \ + --from-literal=PROWLARR_API_KEY=$PROWLARR_API_KEY \ + --from-literal=PREMIUMIZE_OAUTH_CLIENT_ID=$PREMIUMIZE_OAUTH_CLIENT_ID \ + --from-literal=PREMIUMIZE_OAUTH_CLIENT_SECRET=$PREMIUMIZE_OAUTH_CLIENT_SECRET \ + --dry-run=client -o yaml | kubectl apply -f - +``` + +## Install Required Addons + +### Ingress 🌐 + +```bash +minikube addons enable ingress +``` + +### Metrics Server 📊 + +Required for Horizontal Pod Autoscaler (HPA) functionality: + +```bash +minikube addons enable metrics-server +``` + +## Create SSL Certificate for Ingress 🔒 + +Generate and store a self-signed certificate: + +```bash +pip install mkcert +mkcert -install +mkcert "mediafusion.local" + +kubectl create secret tls mediafusion-tls \ + --cert=mediafusion.local.pem \ + --key=mediafusion.local-key.pem +``` + +## Configuring MediaFusion + +Edit the `deployment/local-deployment.yaml` to set the required environment variables: + +```yaml + - name: HOST_URL + value: "https://mediafusion.local" + - name: ENABLE_TAMILMV_SEARCH_SCRAPPER + value: "false" + - name: PROWLARR_IMMEDIATE_MAX_PROCESS + value: "3" + - name: PROWLARR_SEARCH_INTERVAL_HOUR + value: "24" + - name: IS_SCRAP_FROM_TORRENTIO + value: "false" +``` + +## Deployment 🚢 + +Deploy MediaFusion to your local Kubernetes cluster: + +```bash +kubectl apply -f deployment/local-deployment.yaml +``` + +## Accessing MediaFusion 🌍 + +Add an entry to your system's hosts file to resolve `mediafusion.local` to the Minikube IP: + +### Windows + +Open PowerShell as Administrator: + +```powershell +Add-Content -Path C:\Windows\System32\drivers\etc\hosts -Value "$(minikube ip) mediafusion.local" +``` + +### Linux/macOS + +```bash +echo "$(minikube ip) mediafusion.local" | sudo tee -a /etc/hosts +``` + +Now, you can access MediaFusion at [https://mediafusion.local](https://mediafusion.local) 🎉 diff --git a/deployment/deployment.yaml b/deployment/deployment.yaml index 7555f4d..6341318 100644 --- a/deployment/deployment.yaml +++ b/deployment/deployment.yaml @@ -58,10 +58,6 @@ spec: key: PROWLARR_API_KEY - name: HOST_URL value: "https://mediafusion.fun" - - name: ENABLE_SCRAPPER - value: "false" - - name: SCRAPPER_PROXY_URL - value: "http://proxybroker-service:8888" - name: PROWLARR_IMMEDIATE_MAX_PROCESS value: "3" livenessProbe: @@ -131,7 +127,7 @@ kind: Deployment metadata: name: prowlarr-deployment spec: - replicas: 10 + replicas: 5 selector: matchLabels: app: prowlarr @@ -140,15 +136,27 @@ spec: labels: app: prowlarr spec: - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: prowlarr-storage - operator: In - values: - - "true" + securityContext: + fsGroup: 1000 + initContainers: + - name: config-setup + image: curlimages/curl:latest + command: ["/bin/sh", "-c"] + args: + - > + curl -o /config/config.xml https://raw.githubusercontent.com/mhdzumair/MediaFusion/develop/resources/xml/prowlarr-config.xml; + sed -i 's/$PROWLARR_API_KEY/'"$PROWLARR_API_KEY"'/g' /config/config.xml; + chmod 664 /config/config.xml; + echo "Prowlarr config setup complete."; + volumeMounts: + - name: config-volume + mountPath: /config + env: + - name: PROWLARR_API_KEY + valueFrom: + secretKeyRef: + name: mediafusion-secrets + key: PROWLARR_API_KEY containers: - name: prowlarr image: ghcr.io/hotio/prowlarr:latest @@ -172,24 +180,49 @@ spec: name: mediafusion-secrets key: PROWLARR_API_KEY ports: - - containerPort: 9696 + - containerPort: 9696 livenessProbe: exec: command: - - sh - - -c - - 'curl -H "X-API-KEY: $PROWLARR_API_KEY" http://localhost:9696/api/v1/health' + - sh + - -c + - 'curl -H "X-API-KEY: $PROWLARR_API_KEY" http://localhost:9696/api/v1/health' initialDelaySeconds: 60 periodSeconds: 60 failureThreshold: 5 timeoutSeconds: 10 volumeMounts: - - name: prowlarr-storage - mountPath: /config + - name: config-volume + mountPath: /config + - name: setup-indexers + image: apteno/alpine-jq:latest + command: ["/bin/sh", "-c"] + args: + - > + echo "Waiting for Prowlarr to be ready..."; + until curl -f -H "X-API-KEY: $PROWLARR_API_KEY" http://localhost:9696/api/v1/health; do + sleep 5; + done; + echo "Prowlarr is ready, setting up indexers..."; + curl -LO https://github.com/mhdzumair/MediaFusion/raw/develop/resources/json/prowlarr-indexers.json; + jq -c '.[]' prowlarr-indexers.json | while read indexer; do + echo "Adding indexer named: $(echo $indexer | jq -r '.name')"; + curl -H "Content-Type: application/json" -H "X-API-KEY: $PROWLARR_API_KEY" -X POST http://localhost:9696/api/v1/indexer -d "$indexer"; + done; + echo "Indexers setup complete."; + sleep infinity; + env: + - name: PROWLARR_API_KEY + valueFrom: + secretKeyRef: + name: mediafusion-secrets + key: PROWLARR_API_KEY + volumeMounts: + - name: config-volume + mountPath: /config volumes: - - name: prowlarr-storage - persistentVolumeClaim: - claimName: prowlarr-pvc + - name: config-volume + emptyDir: {} --- diff --git a/deployment/local-deployment.yaml b/deployment/local-deployment.yaml index 251823c..5a0c995 100644 --- a/deployment/local-deployment.yaml +++ b/deployment/local-deployment.yaml @@ -25,10 +25,10 @@ spec: - containerPort: 80 resources: requests: - memory: "200Mi" + memory: "500Mi" cpu: "200m" limits: - memory: "300Mi" + memory: "800Mi" cpu: "200m" env: - name: MONGO_URI @@ -54,11 +54,15 @@ spec: name: mediafusion-secrets key: PROWLARR_API_KEY - name: HOST_URL - value: "https://mediafusion.fun" - - name: ENABLE_SCRAPPER + value: "https://mediafusion.local" + - name: ENABLE_TAMILMV_SEARCH_SCRAPPER value: "false" - - name: SCRAPPER_PROXY_URL - value: "http://proxybroker-service:8888" + - name: PROWLARR_IMMEDIATE_MAX_PROCESS + value: "3" + - name: PROWLARR_SEARCH_INTERVAL_HOUR + value: "24" + - name: IS_SCRAP_FROM_TORRENTIO + value: "true" livenessProbe: httpGet: path: /health @@ -71,7 +75,7 @@ spec: httpGet: path: /health port: 80 - initialDelaySeconds: 5 + initialDelaySeconds: 30 periodSeconds: 5 timeoutSeconds: 10 --- @@ -120,7 +124,7 @@ spec: containers: - name: dramatiq-worker image: mhdzumair/mediafusion:beta - command: ["dramatiq", "api.task"] + command: ["pipenv", "run", "dramatiq", "api.task"] env: - name: MONGO_URI value: "mongodb://mongodb-service:27017/mediafusion" @@ -136,10 +140,10 @@ spec: key: PROWLARR_API_KEY resources: requests: - memory: "200Mi" + memory: "300Mi" cpu: "200m" limits: - memory: "300Mi" + memory: "800Mi" cpu: "200m" --- @@ -158,6 +162,27 @@ spec: labels: app: prowlarr spec: + securityContext: + fsGroup: 1000 + initContainers: + - name: config-setup + image: curlimages/curl:latest + command: ["/bin/sh", "-c"] + args: + - > + curl -o /config/config.xml https://raw.githubusercontent.com/mhdzumair/MediaFusion/develop/resources/xml/prowlarr-config.xml; + sed -i 's/$PROWLARR_API_KEY/'"$PROWLARR_API_KEY"'/g' /config/config.xml; + chmod 664 /config/config.xml; + echo "Prowlarr config setup complete."; + volumeMounts: + - name: config-volume + mountPath: /config + env: + - name: PROWLARR_API_KEY + valueFrom: + secretKeyRef: + name: mediafusion-secrets + key: PROWLARR_API_KEY containers: - name: prowlarr image: ghcr.io/hotio/prowlarr:latest @@ -175,15 +200,55 @@ spec: value: "1000" - name: UMASK value: "002" + - name: PROWLARR_API_KEY + valueFrom: + secretKeyRef: + name: mediafusion-secrets + key: PROWLARR_API_KEY ports: - - containerPort: 9696 + - containerPort: 9696 + livenessProbe: + exec: + command: + - sh + - -c + - 'curl -H "X-API-KEY: $PROWLARR_API_KEY" http://localhost:9696/api/v1/health' + initialDelaySeconds: 60 + periodSeconds: 60 + failureThreshold: 5 + timeoutSeconds: 10 volumeMounts: - - name: prowlarr-storage - mountPath: /config + - name: config-volume + mountPath: /config + - name: setup-indexers + image: apteno/alpine-jq:latest + command: ["/bin/sh", "-c"] + args: + - > + echo "Waiting for Prowlarr to be ready..."; + until curl -f -H "X-API-KEY: $PROWLARR_API_KEY" http://localhost:9696/api/v1/health; do + sleep 5; + done; + echo "Prowlarr is ready, setting up indexers..."; + curl -LO https://github.com/mhdzumair/MediaFusion/raw/develop/resources/json/prowlarr-indexers.json; + jq -c '.[]' prowlarr-indexers.json | while read indexer; do + echo "Adding indexer named: $(echo $indexer | jq -r '.name')"; + curl -H "Content-Type: application/json" -H "X-API-KEY: $PROWLARR_API_KEY" -X POST http://localhost:9696/api/v1/indexer -d "$indexer"; + done; + echo "Indexers setup complete."; + sleep infinity; + env: + - name: PROWLARR_API_KEY + valueFrom: + secretKeyRef: + name: mediafusion-secrets + key: PROWLARR_API_KEY + volumeMounts: + - name: config-volume + mountPath: /config volumes: - - name: prowlarr-storage - persistentVolumeClaim: - claimName: prowlarr-pvc + - name: config-volume + emptyDir: {} --- @@ -214,7 +279,7 @@ kind: Deployment metadata: name: proxybroker-deployment spec: - replicas: 1 + replicas: 0 selector: matchLabels: app: proxybroker @@ -250,7 +315,7 @@ spec: - protocol: TCP port: 80 targetPort: 80 - type: LoadBalancer + type: NodePort --- @@ -322,16 +387,19 @@ spec: --- -apiVersion: v1 -kind: PersistentVolumeClaim +apiVersion: autoscaling/v1 +kind: HorizontalPodAutoscaler metadata: - name: prowlarr-pvc + name: mediafusion-hpa + namespace: default spec: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: mediafusion-deployment + minReplicas: 1 + maxReplicas: 3 + targetCPUUtilizationPercentage: 50 --- @@ -355,7 +423,7 @@ spec: cpu: "200m" limits: memory: "256Mi" - cpu: "250m" + cpu: "200m" env: - name: MONGO_URI value: "mongodb://mongodb-service:27017/mediafusion" @@ -364,8 +432,6 @@ spec: secretKeyRef: name: mediafusion-secrets key: SECRET_KEY - - name: SCRAPPER_PROXY_URL - value: "http://proxybroker-service:8888" restartPolicy: OnFailure --- @@ -389,7 +455,7 @@ spec: cpu: "200m" limits: memory: "256Mi" - cpu: "250m" + cpu: "200m" env: - name: MONGO_URI value: "mongodb://mongodb-service:27017/mediafusion" @@ -398,6 +464,29 @@ spec: secretKeyRef: name: mediafusion-secrets key: SECRET_KEY - - name: SCRAPPER_PROXY_URL - value: "http://proxybroker-service:8888" restartPolicy: OnFailure + +--- + +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: mediafusion-ingress + annotations: + nginx.ingress.kubernetes.io/ssl-redirect: "true" +spec: + tls: + - hosts: + - mediafusion.local + secretName: mediafusion-tls + rules: + - host: mediafusion.local + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: mediafusion-service + port: + number: 80