Setting up MLflow with google cloud is an easy process and can be done in a few simple steps from the Google cloud platform.

Bobcares answers all questions no matter the size, as part of our GCP Support

Let us take a look at setting up mlflow on google cloud

MLflow

It is an open-source platform to control the whole machine learning lifecycle. The following are its main functions:

1. Tracking: The user can keep track of experiments so they may log and evaluate their parameters and outcomes.
2. Models: To administer and distribute models from various machine learning libraries to various model serving and inference platforms.
3. Projects: For, package machine learning code in a repeatable, reproducible manner to move to production or share with other data scientists.
4. Model Registry: Create a centralized model store with versioning and annotation tools for managing the whole lifecycle stage transitions of models, from development to production.
5. Model serving: REST endpoints to host MLflow models.

Steps to run MLflow on Google Compute Engine

Set up the MLflow server on Compute Engine by following these steps:

1. Firstly, construct a VM instance on the basis of Ubuntu Linux 18.04 LTS
2. Secondly, Install MLflow on a virtual machine — SSH into the VM using Putty or the console and perform the following instructions to install mlflow. mlflow.

   sudo apt update
sudo apt upgrade
# Install pip
sudo apt install python-pip
# Install mlflow
pip install mlflow
# Check version of mlflow installed
mlflow --version

   Executed commands on the shell

   abcd1234@cloudshell:-(quickstart-1560461844415) $ mlflow --version
/usr/local/lib/python3.7/site-packages/pandas/compat/__init__.py:117: UserWarning:
result in a RuntimeError.
warnings.warn(msg)
mlflow, version 1.7.1

   Checking MLflow version
3. And then, run the MLflow server on the VM instance’s internal IP address (default 127.0.0.1) Using the command ifconfig, check the internal IP address. Its output will be as follows:

   Your cloud Platform project in this session is set to quickstart-1560461844415. Use "gcloud config set project [PROJECT_ID]" to change to a different project. abcd 1234@cloudshell: (quickstart-1560461844415) $ ifconfig dockero: flags=4099<up, broadcast,="" multicast=""> mtu 1500
inet 172.18.0.1 netmask 255.255.0.0 broadcast 172.18.255.255
ether 02:42:4b:d8:e8:ca txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors Odropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors0 dropped 0 overruns O carrier 0 collisions 0

   etho: flags=4163<up, broadcast,="" running,="" multicast=""> mtu 1500
inet 172.17.0.2 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:ac:11:00:02 txqueueleno (Ethernet)
RX packets 160 bytes 40164 (39.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 126 bytes 28451 (27.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

   lo: flags=73<up, loopback,="" running=""> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 226 bytes 25963 (25.3 KiB)
RX errors 0 dropped overruns 0 frame 0
TX packets 226 bytes 25963 (25.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
abcd 1234@cloudshell:- (quickstart-1560461844415) $1

   Check for internal IP using ifconfig

   The internal IP address is 172.18.x.x, as seen above. Use the following command to launch the MLflow server: ‘mlflow server —host’ 172.18.x.x (replace 172.18.x.x with your internal IP):

   abcd 1234@cloudshell: (quickstart-1560461844415) $ mlflow server --host 172.18.0.1 /usr/local/lib/python3.7/site-packages/pandas/compat/__init__.py: 117: UserWarning: Could no result in a RuntimeError.
warnings.warn (msg)
[2020-03-18 01:57:38 +0530] [2814] [INFO] Starting gunicorn 20.0.4
[2020-03-18 01:57:38 +0530] [2814] [INFO] Listening at: http://172.18.0.1:5000 (2814)
[2020-03-18 01:57:38 +0530] [2814] [INFO] Using worker: sync
[2020-03-18 01:57:38 +0530] [2817] [INFO] Booting worker with pid: 2817
[2020-03-18 01:57:38 +0530] [2818] [INFO] Booting worker with pid: 2818
[2020-03-18 01:57:38 +0530] [2819] [INFO] Booting worker with pid: 2819,
[2020-03-18 01:57:38 +0530] [2820] [INFO] Booting worker with pid: 2820

   Start mlflow server on Internal IP
4. After that, Use curl -I to check from the VM instance (from the second connection) at curl -I http://172.18.x.x:5000

   curl -I http://172.18.x.x:5000

   Checking using another connection

   abcd 1234cloudhsell:~ (quickstart-1560461844415) $ curl -I http://172.18.0.1:5000

   HTTP/1.1 200 OK
Server: gunicorn/20.0.4
Date: Tue, 17 Mar 2020 20:32:43 GMT
Connection: close
Content-Length: 853
Content-Type: text/html; charset=utf-8
Last-Modified: Tue, 17 Mar 2020 20:19:43 GMT
Cache-Control: public, max-age=43200
Expires: Wed, 18 Mar 2020 08:32:43 GMT
ETag: "1584476383.459983-853-3330939614"
5. And then, Set the tag for the network as mlflow
6. Next, to provide access to port 5000 create a Firewall Rule.

   gcloud compute --project=test-prj firewall-rules create mlflow-server --direction=INGRESS --priority=999 --network=default --action=ALLOW --rules=tcp:5000 --source-ranges=0.0.0.0/0 --target-tags=mlflow-server

   Firewall Rule to open Port 5000
7. Check from the Linux machine(on-premises) nmap -Pn 32.1.XXX.XXX

   Starting Nmap 7.80 ( https://nmap.org ) at 2020-03-09 16:20 CET
Nmap scan report for x.x.x.x.bc.googleusercontent.com (32.1.XXX.XXX)
Host is up (0.20s latency).
Not shown: 993 filtered ports
PORT STATE SERVICE
...
5000/tcp open upnp
...
8. Finally, head to the web browser http://x.x.x.x:5000/ (external IP) for mlflow to run on google-cloud

Using Python or REST, the MLflow Tracking component enables users to log and query experiments. The user may currently close this mlflow server. After this, the storage bucket must be set up to hold the artifacts.

How to Make a Google Bucket for Storing Artifacts?

Set up the storage bucket for logging models and artifacts by following these steps:

1. Go to the Google cloud platform and click on Create Bucket:
2. Give the storage bucket a name, fill in the other parameters, and then click Create
3. After the bucket creation, open it and navigate to permissions: permissions :

   The user must either add a new member or utilize an existing member with Storage Object Viewer and Storage Object Creator rights.
4. Get the bucket path, which will be something like gs:/// .
5. Finally, To access Google Cloud Storage, run this command on both the client and the server. pip install google-cloud-storage
6. Start the MLflow tracking server by specifying the bucket path as the default artifact directory and spinning off the run server with an internal IP address. By default, it will utilize port 5000:

   mlflow server --backend-store-uri ./mlruns --default-artifact-root gs://mlflow_artifacts/artifacts --host 172.18.x.x

   And then, Run the mlflow server.
7. Now the user can access their server using its public IP http://x.x.x.x:5000/

Artifact Location has now been assigned to the storage bucket path, as indicated above.  This is the final step in mlflow google cloud setup.

[Need assistance with similar queries? We are here to help]

Conclusion

To conclude MLflow is an open-source platform that allows users to manage the whole ML lifecycle via experimentation, deployment, and a common model registry. And, the users can easily set up the MLflow with the google cloud in a few simple steps.