The best tool for the job?
For SRE, any manual, structurally mandated operational task is abhorrent.
As a contract network engineer, I tend to move around a bit. I get to work in lots of different places with lots of different people which is quite fun. Recently, I was asked to configure a few boxes (Juniper SRXs) for a service that was being migrated. In some places, the work changes day by day but occasionally, you start to see the same jobs come around which opens up the inevitable question... "Is there a better way?"
What's my definition of better? Minimum effective dose, as Tim Ferriss says.
Before heading straight to the Ansible (or similar CoolTooling of the moment) download page, it's worth remembering that I could be gone any day and some organisations take months (I tried to get TCPping added to the toolset whitelist in one such place...) to sign off new tooling. This is especially true in roles that demand Security Clearance. It's also worth remembering that junior engineers cut their teeth on jobs like this where visibility of the syntax is essential. To this end, the XML equivalent of the commit is not always desirable.
A rough outline of the service configuration, in this particular, instance was as follows:
# Set up the interface with the appropriate VLAN, IP address and subnet mask:
set interfaces ge-0/0/4 vlan-tagging
set interfaces ge-0/0/4 unit 333 vlan-id 333
set interfaces ge-0/0/4 unit 333 family inet address 10.33.33.1/29
# Add the interface to the relevant VRF:
set routing-instances LOB interface ge-0/0/4.333
# Add the interface and prefix to the address book within the relevant zone:
set security zones security-zone SERVICE1 address-book address SERVICE1NET 10.33.33.0/29
set security zones security-zone SERVICE1 interfaces ge-0/0/4.330
# Create the prefix list and the BGP community:
set policy-options prefix-list SERVICE1NET 10.33.33.0/29
set policy-options community SERVICE1 members 65001:333
# Create the export policy to tag the community:
set policy-options policy-statement Export_to_LOB term SERVICE1 from prefix-list SERVICE1NET
set policy-options policy-statement Export_to_LOB term SERVICE1 then community add SERVICE1
set policy-options policy-statement Export_to_LOB term SERVICE1 then accept
# Create the security policy:
set security policies from-zone SERVICE1 to-zone LOB policy SERVICE1_Access match source-address SERVICE1NET
set security policies from-zone SERVICE1 to-zone LOB policy SERVICE1_Access match destination-address any
set security policies from-zone SERVICE1 to-zone LOB policy SERVICE1_Access match application any
set security policies from-zone SERVICE1 to-zone LOB policy SERVICE1_Access then permit
set security policies from-zone SERVICE1 to-zone LOB policy SERVICE1_Access then log session-init
The same VLAN ID was used for this service at each site which means we have four variables we need to consider:
1. The base interface (ge-0/0/4)
2. The interface IP (10.33.33.1/29)
3. The address-book entry (10.33.33.0/29)
4. The prefix-list entry (10.33.33.0/29)
With the last two being identical, we'll only need to supply three arguments as the input to whichever script we create to generate a complete configuration to be pushed to the device. Part of me immediately wondered if I could just use the first two. It seemed possible the prefix-list entry might take 10.33.33.1/29 as input and automatically convert it to 10.33.33.0/29 for me:
{primary:node0}[edit]
root@msbnet_node0# set policy-options prefix-list SERVICE1NET 10.33.33.1/29
error: host portion is not zero (10.33.33.0/29): 10.33.33.1/29Negative on that, Houston!
Fine! What about the address-book entry, though?
{primary:node0}[edit]
root@msbnet_node0# set security zones security-zone SERVICE1 address-book address SERVICE1NET 10.33.33.1/29
{primary:node0}[edit]
root@msbnet_node0# commit check
[edit security zones security-zone SERVICE1 address-book]
'address SERVICE1NET'
Invalid address entry
error: configuration check-out failedThat's a negative, too, it seems!
So I created a batch script which took those three essential arguments, swapped them into the variables in a template and then echoed the results to a file. That worked well enough for that day as I needed to get those services live ASAP but that night, I wondered how I might be able to get that down to just two arguments and have the script work out the network address itself. I had no idea how I was going to do this but I did have the vaguest recollection, from my CCNA days, bouncing around the back of my head...
Computers deduce their network address by performing a logical AND on the binary equivalent of the IP address.
So I started Googling 'logical AND' and then rummaging through StackOverflow and GitHub. Someone must have solved this problem before...?
After several minutes of ingesting a dizzying array of complex sounding terms, I decided to re-familiarise myself with the basics. From Wikipedia:
Logical conjunction is often used for bitwise operations, where 0 corresponds to false and 1 to true:
0 AND 0 = 0
0 AND 1 = 0
1 AND 0 = 0
1 AND 1 = 1The AND of a set of operands/inputs is true if and only if ALL of it's operands are true.
It was at this point I realised I'd been googling the wrong thing. It seemed what I was actually trying to achieve was a 'bitwise AND'. It is the bitwise AND which takes a normal number (or tiny integer if you prefer), converts it to it's binary form and then performs a logical AND on it. This subtle difference cost me a good few hours!
A quick example for our use case:
An IP (IPv4) address is said to be a 32 bit address, written in 'dot decimal' notation. A better description might be that it's four lots of 8 bit addresses (octets) wedged together by a period/full stop: 10.33.33.1
An 8 bit address simply means there are a maximum of 8 bits or place holders available to represent a number. imagine a table, with 8 fields, that are labelled like this:
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
You can place the digit one or zero in any field. The maximum value we can represent is 255. We would do this by placing the digit one in all eight columns. If we wanted to represent a number higher than 255, we'd need more bits. Going left, each additional field added would double the size of the one that preceded it.
Back to 8 bits. Let's work out, for example, my age in binary. I'm 37.
Starting from the leftmost bit, sometimes referred to as the most significant bit, navigate from left to right until you find a column where your age fits within either perfectly or with a remainder:
128 = no
64 = no
32 = yes, remainder 5!
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
| 1 |
We've now accounted for 32 out of the 37 total years. Where do we put the remaining 5?
16 = no
8 = no
4 = yes, remainder 1!
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
| 1 | 1 |
2 = no
1 = yes, perfect fit!
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
| 1 | 1 | 1 |
Put zeros in any remaining columns:
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
| 0 | 0 | 1 | 0 | 0 | 1 | 0 | 1 |
Now write out the set of zeros and ones and we have my age, in 8 bit binary: 00100101
Armed with this information, let's attempt to, manually, perform a bitwise AND on the IP address we've specified earlier against the subnet mask we also specified; 10.33.33.1/29.
Let's do the IP first. Take each octet in turn and convert it into the binary equivalent.
00001010.00100001.00100001.00000001 = 10.33.33.1
This leaves us with the subnet mask. Sometimes, the subnet mask will be written in the same format as the IP address - dot decimal - and sometimes it will be written in shorthand or 'CIDR' notation. /29 is an example of CIDR notation.
/29 simply means the first 29 bits of the subnet mask, from the most significant bit, are set to one. The remaining bits will be set to zero.
For prefixes longer than 24 bits (the overwhelming majority of all prefixes you're likely to configure on Customer Edge devices), the first three octets will always be 'maxed out'. This means we can skip to the last octet just like we did before. Let's set the first five bits of the last octet to one and see what value that gives us:
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 |
| 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 |
128 + 64 + 32 + 16 + 8 = 248
The calculator appears to agree!
To summarise:
/29 = 8 bits.8 bits.8 bits.5 bits
/29 = 11111111.11111111.11111111.11111000
/29 = 255.255.255.248
Now that we have the binary representation of both, we can attempt the logical AND. We do this by comparing the most significant bit of A with the most significant bit of B with the all or nothing mindset that epitomises a logical AND.
We'll put the result into C. If they're not both 1, the result is 0.
Why are we doing this again? We hope the result of performing a logical AND on A (IP address) and B (subnet mask) will yield the network/base address in C which will save us from having to manually work it out and submit it as a script argument for the next 100 potential sites. We'll use the & operator below to indicate we're performing a logical AND.
A: 00001010.00100001.00100001.00000001 (10.33.33.1)
B: 11111111.11111111.11111111.11111000 (255.255.255.248)
First octet
0&1=0
0&1=0
0&1=0
0&1=0
1&1=1
0&1=0
1&1=1
0&1=0
Second octet
0&1=0
0&1=0
1&1=1
0&1=0
0&1=0
0&1=0
0&1=0
1&1=1
Third octet
0&1=0
0&1=0
1&1=1
0&1=0
0&1=0
0&1=0
0&1=0
1&1=1
Fourth octet
0&1=0
0&1=0
0&1=0
0&1=0
0&1=0
0&0=0
0&0=0
1&0=0
Result
C: 00001010.00100001.00100001.00000000 (10.33.33.0) <------ As expected, the network address is zero in this instance. The theory holds!
Now, the actual work can begin. Let's remind ourselves of the desired outcome. I want to run a script and specify the absolute minimum number of arguments in order to generate the configuration for SERVICE1. Something like:
script.bat <ARG1> <ARG2>
SERVICE1.bat ge-0/0/4 10.33.33.1/29
The quickest way forward now would be to create a first draft of sorts that simply accepts a prefix as input and then echoes the subsequent network address back to us.
@echo off set ADDR="%1" set C=255.255.255. for /f "tokens=4 delims=./" %%a in (%ADDR%) do set OCTET4=%%a for /f "tokens=1,2,3 delims=." %%x in (%ADDR%) do set OCTET123=%%x.%%y.%%z. for /f "tokens=2 delims=/ " %%m in (%ADDR%) do set SLASH=%%m set /a MASKOCTET4="255 - (255 >> (%SLASH%-24))" set SUBNETMASK=%C%%MASKOCTET4% set /a SUBN="%OCTET4% & %MASKOCTET4%" echo. echo NETWORK ADDRESS: %OCTET123%%SUBN%/%SLASH% echo SUBNET MASK: %SUBNETMASK%
Save this file with a .bat extension (prefix.bat) and then call it from a command prompt:
C:\Users\Michael>prefix 10.33.33.1/29 NETWORK ADDRESS: 10.33.33.0/29 SUBNET MASK: 255.255.255.248
Job done!
So what exactly are we doing here? Essentially, we chop up the prefix into more usable chunks before re-assembling it and spitting it out at the end. We also perform a logical shift (where, using the table above, we simply fast forward over the bits) to calculate MASKOCTET4 using the CIDR notation from the prefix to calculate how many bits to shift right. Finally, we perform the bitwise AND on the fourth octet of the prefix vs the fourth octet of the calculated subnet mask.
The easiest way to see what's going on is to echo the variables out as we're setting / calculating them:
@echo off echo. set ADDR="%1" set C=255.255.255. for /f "tokens=4 delims=./" %%a in (%ADDR%) do set OCTET4=%%a echo OCTET4: %OCTET4% for /f "tokens=1,2,3 delims=." %%x in (%ADDR%) do set OCTET123=%%x.%%y.%%z. echo OCTET123: %OCTET123% for /f "tokens=2 delims=/ " %%m in (%ADDR%) do set SLASH=%%m echo SLASH: %SLASH% set /a MASKOCTET4="255 - (255 >> (%SLASH%-24))" echo MASKOCTET4: %MASKOCTET4% set SUBNETMASK=%C%%MASKOCTET4% set /a SUBN="%OCTET4% & %MASKOCTET4%" echo SUBN: %SUBN% echo. echo NETWORK ADDRESS: %OCTET123%%SUBN%/%SLASH% echo SUBNET MASK: %SUBNETMASK%
Save this file with a .bat extension (prefix_debug.bat) and then call it from a command prompt and try a few prefixes:
C:\Users\Michael>prefix_debug 10.33.33.1/29 OCTET4: 1 OCTET123: 10.33.33. SLASH: 29 MASKOCTET4: 248 SUBN: 0 NETWORK ADDRESS: 10.33.33.0/29 SUBNET MASK: 255.255.255.248 C:\Users\Michael>prefix_debug 10.33.33.1/28 OCTET4: 1 OCTET123: 10.33.33. SLASH: 28 MASKOCTET4: 240 SUBN: 0 NETWORK ADDRESS: 10.33.33.0/28 SUBNET MASK: 255.255.255.240 C:\Users\Michael>prefix_debug 10.33.33.1/27 OCTET4: 1 OCTET123: 10.33.33. SLASH: 27 MASKOCTET4: 224 SUBN: 0 NETWORK ADDRESS: 10.33.33.0/27 SUBNET MASK: 255.255.255.224 C:\Users\Michael>prefix_debug 10.33.33.103/29 OCTET4: 103 OCTET123: 10.33.33. SLASH: 29 MASKOCTET4: 248 SUBN: 96 NETWORK ADDRESS: 10.33.33.96/29 SUBNET MASK: 255.255.255.248 C:\Users\Michael>prefix_debug 10.33.33.221/27 OCTET4: 221 OCTET123: 10.33.33. SLASH: 27 MASKOCTET4: 224 SUBN: 192 NETWORK ADDRESS: 10.33.33.192/27 SUBNET MASK: 255.255.255.224
A few minutes later, we have a script that takes just the two essential arguments and will run on any Windows box without installing any additional software. A far cry from full blown automation but infinitely more scalable than find and replace?