Job Activity Log
| Server | Type | Status | Step | Started | Duration |
|---|---|---|---|---|---|
Infrastructure Automation Microservice
System overview and job monitoring
No active operations
| Server | Type | Status | Step | Started | Duration |
|---|---|---|---|---|---|
| Loading... | |||||
Test API endpoints with live requests
Service → LibreNMS bill mappings. Preview computes the current/previous overage; Run (dry) previews the WHMCS push; Run (charge) pushes it.
Map a WHMCS service to a LibreNMS traffic bill (adopt an existing bill or auto-create one). Pick the client, the service, and the interface(s) — no id typing.
Read the previous (closed) LibreNMS period, compute overage, and (unless dry-run) push it to WHMCS. Idempotent per (service, period).
WHMCS service → per-token LLM billing. Grant trial credit, preview the period, and run billing (idempotent per period). Usage is metered from the inference proxy.
Map a WHMCS service to per-token pricing. Grant a trial credit ($ of free usage) to attract the client; enable "enforce credit limit" so a trial can't overrun before they add a payment method.
Aggregate the previous (closed) period's tokens, apply trial credit, and (unless dry-run) push the net overage to WHMCS. Idempotent per (service, period).
Register Proxmox endpoints (a standalone node or a per-site cluster). Nodes can't be clustered across sites, so each location is its own endpoint. The API token secret is encrypted at rest (never shown). Test probes /version + node-online live.
Clone a golden template on a registered node, optionally attach whole GPUs by PCI bus address, put the service NIC on a subnet, claim one host IP, cloud-init it. GPUs optional — serves GPU and plain VMs alike. Runs as a background job.
Look up a VM's NetBox record + live Proxmox status, or tear it down (destroy the Proxmox VM + free its NetBox IPs). Deprovision requires typed confirmation.
Live GPU pool (passthrough-mapped, free/assigned) + last burn-in result per GPU. Sync reflects the pool into NetBox inventory items. Burn-in provisions a throwaway 1-GPU VM, runs gpu-burn, scores PASS/FAIL, and deprovisions — sequential per node.
Configure a switch port for Foreman bare metal provisioning. This looks up the server in NetBox, traces the cable to find the connected switch port, saves the current configuration, and sets the port to access mode with the Foreman VLAN.
{
"device_label": "server-001"
}
{
"device_label": "bm-web-prod-01",
"vlan_id": 100,
"description": "Foreman PXE - OS Reinstall",
"service_id": "12345",
"requested_by": "whmcs-foreman"
}
curl -X POST "http://localhost:8000/api/v2/port/provision" \
-H "X-API-Key: your-api-key" \
-H "Content-Type: application/json" \
-d '{"device_label": "server-001"}'
Restore a switch port to its original configuration after provisioning is complete. This finds the backup, restores the original configuration, and marks the backup as restored.
Get the current status of a switch port and any associated backups.
List every VLAN NetBox has for the device — each labeled native (untagged) or tagged — plus existing subnets reusable for a new VLAN.
Assign one or more VLANs from the site's customer VLAN group to a server's bond trunk and (optionally) build the switch. One VLAN may be native; each VLAN may auto-allocate a subnet, reuse an existing one, or be L2-only. Limited to 5 VLANs (QinQ beyond that).
Create ONE private (RFC1918/VRF, L2-only) VLAN and assign it to several selected servers in one call — e.g. a Proxmox cluster's private network. All nodes must share a switch (cross-switch selections are refused — inter-switch trunking to the private core is a later phase). Each node's bond trunk is rebuilt as a background job.
The ONLY endpoint that changes which VLAN is native (untagged/management) on a server's bond0 — assign/cluster/attach never touch the native. The target VLAN must already be a bond0 member. Demotes the old native to tagged, promotes the target, and rebuilds the switch. Changing the native can interrupt management access (IPMI-level recovery if it goes wrong) — use carefully.
One screen to onboard a hand-racked server: model the bond (eth0/eth1→bond0), stamp NIC MACs from the BMC over Redfish (no PXE needed) with a 10G confirmation check, record the IPMI address + admin credentials (ARPHost-Private VRF + custom fields), and reflect the live switch trunk's VLANs into NetBox. Optionally auto-create the cabling from the rack standard. Always dry-run first — the preview makes zero writes.
ipmi interface (ARPHost-Private VRF) and set as the device Out-of-band IP. Bare host follows the rack's subnet (default /26). Leave blank to reuse the IPMI address/creds already in NetBox.ipmi_admin_username/ipmi_admin_password custom fields. Never echoed back.Release/delete customer VLANs from a server's bond: removes membership, frees IPs/prefixes, deletes the VLAN records, and tears down the switch (reverts the ae to LACP-only when the last VLAN is removed). Use this instead of admin IP release for bonded VLAN devices.
Add one address family's subnet (prefix + gateway + server host IP + Network/Broadcast) to an existing VLAN on a server's bond, then build the irb gateway on the switch.
Delete just one address family's subnet (host IPs + gateway + prefix) from a VLAN on a server's bond — keeps the VLAN + the other family — and drops irb.{vid} family inet/inet6 on the switch.
Create a tenant-owned shared VLAN + public subnet that multiple servers can attach to. No server is attached yet — use Attach afterward.
Allocate a v6 /64 (gateway ::1) for an existing shared VLAN and rebuild attached servers' switch so irb.{vid} gets its v6 gateway. Use the vlan_netbox_id (not the VID). 400 if it already has IPv6.
Show a shared VLAN's subnet, attached servers, and IP usage.
Add a server to the shared VLAN — claims the next host IP from the shared subnet and builds that server's trunk.
subnets[] from the shared-VLAN detail.Remove a server from the shared VLAN — frees just that server's IP and rebuilds its trunk. The VLAN+subnet is deleted only when the last server detaches.
List all port configuration backups with optional filtering by status and device.
Get detailed information about a specific configuration backup.
Delete a backup record from the database. This does NOT restore the port configuration.
Basic health check endpoint. No authentication required.
Query the current power state of your server using tenant IPMI credentials.
Reboot your server using tenant IPMI credentials. Requires a safety confirmation string.
Power on your server using tenant IPMI credentials. This is a safe, non-destructive operation.
Power off your server using tenant IPMI credentials. Supports graceful (ACPI) and force (hard) shutdown modes.
Update your server's IPMI credentials. The new credentials will be tested against the BMC before being saved.
Query the current power state of a server via IPMI/BMC. Returns whether the chassis is powered on, off, or unknown.
Reboot a server using IPMI/BMC out-of-band management. Requires a safety confirmation string.
Power on a server that is currently off via IPMI/BMC. If the server is already on, this has no effect. This is a safe, non-destructive operation that does not require confirmation.
Power off a server via IPMI/BMC. Supports graceful (ACPI) and force (hard) shutdown modes.
Query server hardware information via IPMI FRU data. This is a read-only out-of-band query that works even when the server is powered off. Returns basic hardware info as reported by the BMC (varies by vendor). Does not update NetBox.
POST /api/v2/baremetal/lifecycle/{device}/test which boots the server and collects detailed OS-level inventory.
List servers available for sale from inventory. Only returns servers with status "inventory" in NetBox, "available-for-sale" tag, that have been wiped and passed hardware testing.
Active self-serve holds. Each row locks a device (a table-only hold — the device stays at NetBox status "inventory") until it is assigned to a client or released. This list also feeds the Assign and Release selectors.
Place an admin hold on an available server (no WHMCS order). The hold is a table row — the device stays "inventory" but is excluded from availability until released or assigned.
Permanently assign a server to a WHMCS client/service: stamps the device tenant + WHMCS ids, sets NetBox status to "active", and deletes any outstanding reservation. Requires the typed confirmation ASSIGN-{device_label}.
Cancel a hold — deletes the reservation row so the device is available for sale again (no NetBox status change).
Release a server back to the available inventory pool. This performs: 1) Disables all switch ports (physical + bond/ae interfaces), 2) Powers off the server (hard power off via IPMI), 3) Updates NetBox device status to "inventory".
Boot server into discovery mode to collect hardware inventory. The server will PXE boot into a discovery image that collects CPU, memory, storage, network interfaces, and IPMI/BMC details.
SSH into the target server and run these commands as root:
1. Download the discovery script:
curl -o /tmp/hardware-discovery.sh http://foreman.arphost.com/pub/hardware-discovery.sh && chmod +x /tmp/hardware-discovery.sh
2. Run discovery:
Or as a single command:
Requires: dmidecode, smartmontools, nvme-cli, pciutils, jq, curl. storcli is auto-downloaded if a RAID controller is detected.# RHEL/CentOS/AlmaLinux/Rocky/FDI: rpm -i http://foreman.arphost.com/pub/storcli/storcli_noarch.rpm # Debian/Ubuntu/Proxmox: curl -o /tmp/storcli.deb http://foreman.arphost.com/pub/storcli/storcli_all.deb && dpkg -i /tmp/storcli.deb
Look up an active discovery job by server MAC address. Used internally by hardware-discovery.sh on FDI when kernel cmdline parameters are not available. Returns the callback_url and job_id for the discovery script to use.
List all operating systems available in Foreman for provisioning.
Use the id or title from the response as the os_override parameter when provisioning.
List all host groups available in Foreman for provisioning.
Use the name or title from the response as the hostgroup parameter when provisioning.
Get comprehensive status from all integrated systems: NetBox (device info, lifecycle state, IPs, interfaces), Foreman (build status, host group), WHMCS (service status, client info), IPMI (current power state), and active jobs.
Provision a server with an operating system. The server must have NetBox status "inventory" to prevent accidental reinstallation of production servers. This workflow configures switch ports, sets up Foreman, PXE boots the server, and waits for the installation callback.
Securely wipe all disks on a server. Boots server into wipe environment and securely erases all disks. After successful wipe, hardware testing is automatically triggered unless skip_test=true.
Run hardware tests on a server. Boots into FDI test environment and runs CPU stress test (stress-ng), memory test (memtester), SMART health check, disk I/O benchmark (fio, full/burn-in only), and network link verification (ethtool). On success, server is automatically returned to inventory unless auto_return_to_inventory=false.
SSH into the target server and run these commands as root:
1. Download the test script:
curl -o /tmp/hardware-test.sh http://foreman.arphost.com/pub/hardware-test.sh && chmod +x /tmp/hardware-test.sh
2. Run test:
Or as a single command:
Requires: stress-ng, fio, memtester, smartmontools, nvme-cli, ethtool, jq, curl (tools installed automatically via dnf if missing)List all baremetal workflow jobs with optional filtering by status, workflow type, and device.
Get detailed information about a specific baremetal job including step-by-step logs and progress.
Cancel a pending or running baremetal job. Only jobs with status "pending" or "running" can be cancelled.
Purge completed job history to remove sensitive data like passwords.
By default, only purges jobs with status "completed", "failed", or "cancelled".
Use include_all=true to purge ALL jobs including pending/running ones.
This is a destructive operation - purged jobs cannot be recovered.
Manually send a wipe completion callback for testing. This simulates what hardware-wipe.sh
sends when disk wipe completes. Use leave_powered_on=true to prevent the server
from being powered off after callback processing.
Manually send a discovery completion callback for testing. This simulates what hardware-discovery.sh
sends when hardware discovery completes. Use leave_powered_on=true to prevent the server
from being powered off after callback processing.
Allocate IPv4 AND IPv6 addresses from site's prefix pools. Creates a dedicated IPv4 prefix (/31 point-to-point up to a /27 block) plus an IPv6 /64, with the network, gateway, server and broadcast IPs populated. Pool selection is automatic: primary pool for new servers, secondary for additional IPs.
Assign a specific (already allocated) IP to a device interface. Use for manual IP assignment or moving IPs between servers.
Release a server's IP allocation back to the pool. Deletes IP addresses assigned to the device and clears primary IP references.
Change which IP is the device's primary. Updates NetBox primary IP pointer and all related descriptions. No switch changes.
Only non-primary IPv4 addresses are shown
Route a secondary subnet to a next-hop IP via static route. Removes gateway from switch interface, adds static route and prefix-list entry. All IPs in the subnet become client-manageable.
Must be a secondary prefix assigned to the device
Must be an IP assigned to the device (typically in the primary subnet)
Restore a statically routed subnet back to interface mode. Re-adds gateway IP to switch interface, removes static route.
Must be a currently routed prefix for the device
List all users in the system with their API key counts. Requires admin privileges and JWT authentication.
Get detailed information about a specific user including their API keys. Requires admin privileges.
Update a user's account settings such as active status and admin privileges. Note: You cannot revoke your own admin status or deactivate your own account.
List all API keys for a specific user. Requires admin privileges.
Update an API key's settings including active status and scopes.
Permanently delete an API key for any user. This cannot be undone.
Create a new user account. The user can be assigned to a tenant and given admin privileges.
Change any user's password. The user will need to use the new password on their next login.
Create an API key for any user. The full key is shown only once!
Generate a temporary token to act as another user. The token expires in 1 hour and cannot be refreshed. Cannot impersonate yourself or other admins.
View audit logs of admin actions. Logs auto-delete after 60 days.
Synchronize tenants and users from NetBox. This imports tenants and their contacts as users, creating API keys with appropriate default scopes.
This will sync all tenants from NetBox and create/update users based on contacts assigned to each tenant. Existing users will be updated, new users will be created with passwords stored in NetBox.
Pull active WHMCS clients and create/update NetBox tenants (with the
whmcs_client_id custom field). Idempotent; never deletes. Run
/tenants/sync-netbox afterward to bring them into the Core API DB.
Map EXISTING NetBox devices to their WHMCS client: sets device tenant +
whmcs_service_id/whmcs_client_id and tenant-owns the
service's existing IPs, so synced tenant users see the devices they own in
WHMCS. Never creates devices or IPs; unmatched services and
conflicts are reported. Run /tenants/sync-whmcs first so tenants exist.
Model Active WHMCS "Virtual Servers" as NetBox virtual machines: creates a
per-hypervisor cluster (from the WHMCS server_name) + the VM
(tenant-owned, with its IPs), or updates an existing one to current.
Creates VMs + missing IPs. Services with no hypervisor
(e.g. Cloudflare) are skipped. Run /tenants/sync-whmcs first so
tenants exist; use /reconcile-whmcs-vms for cancellations.
Clean up synced VMs whose WHMCS service left Active: Suspended
→ VM set offline (kept); Terminated/Cancelled/absent
→ VM deleted + its IPs freed (unassigned + tenant cleared, IP
record kept). Manually-created VMs (no whmcs_service_id) are never
touched. Skips on an empty WHMCS service set (safety).
Manage client-private RFC1918 space. Invariant: the global table holds only public space; all RFC1918 lives in a per-client VRF (ARPHost included). All actions are dry-run by default and never create devices.
Lists any RFC1918 still in the global table (should be empty after migration).
Sweeps existing global RFC1918 (install-networks, infra, OOB/IPMI) into the ARPHost VRF.
Paste the backfill request JSON (tenant + VLAN specs + per-server host IPs). Dry-run first, review, then uncheck dry_run.
Reconcile a duplicate tenant: reassign the source tenant's NetBox objects (devices/IPs/prefixes/VLANs) and Core API DB users to the target, then delete the source. Use the NetBox tenant IDs. Run a dry-run first.
View all API activity for users in your tenant. Logs auto-delete after 60 days. Only tenant admins can view these logs.
View all devices assigned to your tenant from NetBox. Shows server details including status, platform, rack location, and IP information.
Change your own password. You must provide your current password for verification.
Set an initial Core API password. Use this if you signed in via WHMCS/Google SSO and have no local password yet. This password is independent of SSO — both work afterward. If you already have a password, use "Change Password" instead.
List all your API keys. You can also create new keys and revoke existing ones.
View, create, and delete IPv4 PTR (reverse DNS) records for your IP addresses.
View, create, and delete IPv6 PTR (reverse DNS) records for your IP addresses.
Preview the reverse DNS name for any IP address. Shows the in-addr.arpa (IPv4) or nibble-format ip6.arpa (IPv6) name and whether a matching reverse zone exists.
List all reverse DNS zones configured on the PowerDNS server.
Create a new reverse DNS zone. Examples: 20.109.78.in-addr.arpa (IPv4 /24),
c.d.4.f.2.0.6.2.ip6.arpa (IPv6 /32).
List all PTR records for IPs assigned to a specific device. Also shows IPs without PTR records.
The default/standard reverse-DNS record every static IP gets. A deleted PTR reverts to this (never blank), and IP assignment fills it in. Edit it here without a code deploy. Leave a field blank to revert it to the env default.
{ip_dashed} and {facility} ({base_domain} optional)netbox-site-slug:facilityView and manage your IPv4 address space. Select a prefix to see all addresses, claim available IPs, and update descriptions.
View and manage your IPv6 address space. Select a prefix to see existing addresses, claim IPs, and update descriptions.
Manage users and their access
| ID | Name | Tenant | Status | Role | Actions | |
|---|---|---|---|---|---|---|
| Click "Refresh" to load users | ||||||
Loading...
Key Created! Copy this now - it won't be shown again:
Manage users in your tenant
| ID | Name | Status | Role | Scopes | Actions | |
|---|---|---|---|---|---|---|
| Loading users... | ||||||
Loading API keys...
Key created! Copy now — won't be shown again:
Loading...
Manage your API keys for programmatic access.
Key created! Copy this now - it won't be shown again:
Loading...
Manage reseller client accounts and resource assignments
| ID | Name | Status | Scopes | Resources | Actions | |
|---|---|---|---|---|---|---|
| Loading sub-users... | ||||||
Loading API keys...