Learn more about Apache NiFi Docker Compose from our experts. Our Apache Support team is here to help you with your questions and concerns.

Apache NiFi Docker Compose | All About

Did you know that NiFi Registry helps with managing data flows and configurations?

Today, we are going to explore the steps to persist NiFi Registry data by mounting volumes, optimizing settings, and more.

Mounting Volumes for Persistence:

NiFi Registry uses two main volumes for data persistence – `flow_storage` and `database`. The former includes buckets and flows, identified by universally unique identifiers.

The directory structure with a bucket inside follows a certain. Additionally, the `database` directory houses a single file for loading buckets and flows from `flow_storage`.

Here is a sample directory structure:

For persistent storage, these volumes are mounted to the local machine. Regardless of container restarts or recreations, the files remain intact outside of the container’s influence.

How to set up NiFi Registry

If we have Docker up and running, and specific ports are available, we can easily create the necessary volumes for persistence as seen below:

sudo mkdir -p /opt/nifi
sudo chown -R 1000:1000 /opt/nifi
sudo mkdir -p /opt/dockerfiles/nifi
sudo vi /opt/dockerfiles/nifi/docker-compose.yml


Then, we have to add the following configuration to the `docker-compose.yml` file:

version: '3'
services:
nifi:
cap_add:
- NET_ADMIN # low port bindings
image: apache/nifi
container_name: nifi
ports:
- "8080:8080/tcp" # HTTP interface
- "8443:8443/tcp" # HTTPS interface
- "514:514/tcp" # Syslog
- "514:514/udp" # Syslog
- "2055:2055/udp" # NetFlow
volumes:
- /opt/nifi/drivers:/opt/nifi/nifi-current/drivers
- /opt/nifi/certs:/opt/certs
- nifi-conf:/opt/nifi/nifi-current/conf
restart: unless-stopped
volumes:
nifi-conf:

Then, bring up the containers:
cd /opt/dockerfiles/nifi/
sudo docker-compose up -d


How to Configure NiFi Bootstrap for More

We can optimize performance by adjusting NiFi’s memory settings:

sudo docker exec -u 0 -it nifi bash
apt-get install vim # Install text editor if needed
vi /opt/nifi/nifi-current/conf/bootstrap.conf


We have to edit the following lines for JVM memory settings:

# JVM memory settings
#java.arg.2=-Xms512m
#java.arg.3=-Xmx512m
java.arg.2=-Xms2g
java.arg.3=-Xmx2g


Then, exit the container, and restart NiFi:

exit
sudo docker restart nifi


Enabling SSL and Authentication

For added security, set up SSL and authentication using NGINX:

sudo mkdir -p /opt/nginx/conf /opt/nginx/keys /opt/nifi/drivers /opt/dockerfiles/nifi/
cd /opt/nginx/keys
sudo openssl req -x509 -sha256 -newkey rsa:2048 -keyout certificate.key -out certificate.crt -days 1024 -nodes -subj '/CN=my.domain.internal'
sudo vi /opt/nginx/conf/nginx.conf


Then, add NGINX configuration:

events {
}
http {
server {
listen 8443 ssl;
server_name my.domain.internal;
ssl_certificate /keys/certificate.crt;
ssl_certificate_key /keys/certificate.key;
proxy_ssl_certificate /keys/certificate.crt;
proxy_ssl_trusted_certificate /keys/certificate.key;
location /nifi {
proxy_pass http://nifi:8080;
proxy_set_header Host $http_host;
proxy_set_header X-ProxyScheme https;
auth_basic "NiFi Authentication";
auth_basic_user_file /keys/nginx.htpasswd;
}
}
}


Now, we have to generate a password for user authentication:

htpasswd -n admin > /opt/nginx/keys/nginx.htpasswd
cat /opt/nginx/keys/nginx.htpasswd # Check the file for output


Finally, it is time to bring up the Docker containers:

cd /opt/dockerfiles/nifi
docker-compose up -d


With the above steps, we can successfully configure NiFi Registry for persistent storage, optimize its performance, and boost security with SSL and authentication.

[Need assistance with a different issue? Our team is available 24/7.]

Conclusion

In brief, our Support Experts introduced us to Apache NiFi Docker Compose and more.