Vector — Self-Host Setup Guide

Vector will accept OTLP HTTP logs on port 4318 and forward to Kafka using your configured bootstrap servers. Follow these steps to get Vector running on your instance for LogWise. This guide extracts only the actionable setup steps.

Prerequisites

Install the following on your instance:

• Vector: Installation Guide (choose your operating system)
• Protocol Buffers compiler (protoc): Installation Guide (only required at build time, not runtime)

Network Configuration

Vector needs to be accessible on port 4318 (OTLP HTTP) to receive logs from OpenTelemetry collectors.

Cloud Instances (AWS/GCP/Azure)

Ensure your security group or firewall rules allow inbound traffic on port 4318:

• AWS: Add inbound rule in Security Group for port 4318 (TCP) from your source IPs
• GCP: Add firewall rule allowing tcp:4318 from your source network
• Azure: Add inbound security rule for port 4318 in Network Security Group

Self-Hosted/On-Premises

Configure your firewall to allow incoming connections on port 4318:

# Example: UFW (Ubuntu/Debian)
sudo ufw allow 4318/tcp

# Example: firewalld (RHEL/CentOS)
sudo firewall-cmd --permanent --add-port=4318/tcp
sudo firewall-cmd --reload

1) Clone repository

git clone https://github.com/ds-horizon/logwise.git
cd logwise/vector/

2) Set Kafka broker address

Edit vector.yaml directly and replace the environment variable with your Kafka address:

# Before
bootstrap_servers: "${KAFKA_BROKERS}"

# After
bootstrap_servers: "<YOUR_KAFKA_ADDRESS>:<KAFKA_PORT>"

3) Compile the protobuf descriptor

From the logwise/vector/ directory (where you are after Step 1), run:

protoc --include_imports --descriptor_set_out=logwise-vector.desc logwise-vector.proto

Important: The protobuf schema defined in logwise-vector.proto uses the package logwise.vector.logs with message type VectorLogs. Ensure that your Spark jobs are configured to consume messages with this schema. If you're upgrading from an older version, you may need to regenerate the Spark protobuf classes and update the Spark code to match the new schema fields (for example: service_name).

4) Install Vector configuration

sudo mkdir -p /etc/vector
sudo cp vector.yaml /etc/vector/vector.yaml
sudo cp logwise-vector.desc /etc/vector/logwise-vector.desc

5) Run Vector

Start Vector as a service (recommended) or run in the foreground for development/testing.

# Production: systemd service (Linux)
sudo systemctl start vector
sudo systemctl enable vector

# Development: run manually
vector --config /etc/vector/vector.yaml

6) Verify

Check that Vector is up and processing logs.

# Service status
sudo systemctl status vector

# Live logs
sudo journalctl -u vector -f

If Vector starts successfully and shows no connection errors, your instance is ready to receive OTLP logs (HTTP 4318) and forward them to Kafka.