Hi,

probably more a "FYI" for anyone who is also hitting the same issue we had. For some reason BIG-IP 15.1.6.1 Build 0.0.10 Point Release 1 has some odd behaviour if one requests an authentication token and uses basic authentication to retrieve that token. It takes over 30 seconds for this request to complete! Funny part is that it works fast if one access the REST API via localhost / 127.0.0.1 (think ssh port forwarding). So I still think that this is just a bug in f5 because it worked with 14.x. (Using 1.4.0 but happens with 1.7.3 too.)

Anyway, the issue can be easily fixed by patching BIGREST: You need to edit big.py and utils/utils.py. Just search for "Required for 12.0.0, but not in 15.1.0".

# ---
# big.py

         # I guess those two lines can be commented also but didn't test (yet).
         if self.request_token is False:
            self.session.auth = (username, password)

         ...
  
         # Required for 12.0.0, but not in 15.1.0
         self.session.auth = (self.username, self.password)  # <= comment this line
         response = None
  
# ---
# utils.py
  
      response = requests.post(
        f"https://{device}/mgmt/shared/authn/login",
        json=data, verify=False, auth=auth)
  
# =>  
  
      response = requests.post(
        f"https://{device}/mgmt/shared/authn/login",
        json=data, verify=False)

Maybe I will provide some patch sometime. But I hope that information is helpful anyway for someone.

Kind regards,
Dietmar

diff -x '*.pyc' -x '*.bak' -r REDACTED/big.py ./big.py
439c439
<         # self.session.auth = (self.username, self.password)
---
>         self.session.auth = (self.username, self.password)
diff -x '*.pyc' -x '*.bak' -r REDACTED/utils/utils.py ./utils/utils.py
57c57
<         json=data, verify=False)
---
>         json=data, verify=False, auth=auth)
88c88
<         json=data, verify=False)
---
>         json=data, verify=False, auth=auth)
[REDACTED@REDACTED bigrest]$

When attempting to run device.command() i get Found invalid content-type no matter what the command is. Here is an example

>>> from pprint import pprint
>>> from bigrest.utils.utils import rest_format
>>> from f5lib import bigrest_login
>>> br = bigrest_login('myF5Device','','')[0]
>>> from bigrest.bigip import BIGIP
>>> cmd = {}
>>> cmd["command"] = "run"
>>> cmd["utilCmdArgs"] = "-c 'cat /VERSION'"
>>> pprint(cmd)
{'command': 'run', 'utilCmdArgs': "-c 'cat /VERSION'"}
>>> br.command("/mgmt/tm/util/bash", cmd)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/bigrest/big.py", line 270, in command
    obj = self.create(path, data)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/bigrest/big.py", line 186, in create
    raise RESTAPIError(response, self.debug)
bigrest.common.exceptions.RESTAPIError:
Status:
415
Response Body:
{
    "code": 415,
    "message": "Found invalid content-type. The content-type must be application/json. The received content-type is application/octet-stream",
    "errorStack": [],
    "apiError": 1
}

I don't see any way to adjust the content-type for the command method. Any advice?

Hi,
I have seen that the connection timeout is set to default "None" which is trying connecting endless when a host is not available.
How can we set the timeout globally for BIGREST?

Thanks

I figure we'll start with the error and everything I'm using is what you have as current :) This is the letsencrypt stuff Jason Rahm's been working on btw. But I'm thinking this issue might be w/ the BIGREST framework and potentially because I'm on F5's BIG-IP 16.1.2.1 Build 0.0.10 Point Release 1. We love how F5 changes stuff all the time ;)

File "/shared/letsencrypt/hook_script.py", line 62, in clean_challenge
vip.properties['rules'].remove('/Common/le_challenge_rule')
ValueError: list.remove(x): x not in list
ERROR: clean_challenge hook returned with non-zero exit code

I see the files created on my Apache server using "watch ls" since it creates then removes them quickly. They did happen so that's working.
The iRule fails to delete per the error and it remains on the box. I've deleted it manually and re-run various times trying various things and still no luck.
I've commented out a few sections of the hook script and it still fails but it gives some insight. I'm not going to explain that now unless ya'll ask.

The other thing I'm seeing and this is more for Jason but this might be the issue w/ the iRule deletion failure.
The tokens seem to mismatch from what's in the log in the CLI vs. the token stored in the challenge iRule.
Thoughts?
Here's a full log example below:

INFO: Using main config file /shared/letsencrypt/config

Processing projectbaiu.org with alternative names: www.projectbaiu.org

• Signing domains...
• Generating private key...
• Generating signing request...
• Requesting new certificate order from CA...
• Received 2 authorizations URLs from the CA
• Handling authorization for www.projectbaiu.org
• Handling authorization for projectbaiu.org
• 2 pending challenge(s)
• Deploying challenge tokens...
• (hook) Deploying Challenge
• (hook) Challenge rule added to virtual.
• (hook) Deploying Challenge
• (hook) Challenge rule added to virtual.
• Responding to challenge for www.projectbaiu.org authorization...
• (hook) Invalid Challenge
• (hook) Invalid Challenge Args: ['www.projectbaiu.org', '["type"]\t"http-01"\n["status"]\t"invalid"\n["error","type"]\t"urn:ietf:params:acme:error:unauthorized"\n["error","detail"]\t"63.226.21.50: Invalid response from http://www.projectbaiu.org/.well-known/acme-challenge/oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ: 404"\n["error","status"]\t403\n["error"]\t{"type":"urn:ietf:params:acme:error:unauthorized","detail":"63.226.21.50: Invalid response from http://www.projectbaiu.org/.well-known/acme-challenge/oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ: 404","status":403}\n["url"]\t"https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/2306706054/wc9l2A"\n["token"]\t"oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ"\n["validationRecord",0,"url"]\t"http://www.projectbaiu.org/.well-known/acme-challenge/oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ"\n["validationRecord",0,"hostname"]\t"www.projectbaiu.org"\n["validationRecord",0,"port"]\t"80"\n["validationRecord",0,"addressesResolved",0]\t"63.226.21.50"\n["validationRecord",0,"addressesResolved"]\t["63.226.21.50"]\n["validationRecord",0,"addressUsed"]\t"63.226.21.50"\n["validationRecord",0]\t{"url":"http://www.projectbaiu.org/.well-known/acme-challenge/oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ","hostname":"www.projectbaiu.org","port":"80","addressesResolved":["63.226.21.50"],"addressUsed":"63.226.21.50"}\n["validationRecord"]\t[{"url":"http://www.projectbaiu.org/.well-known/acme-challenge/oQL3Lr39uV-G-Bp4xq54VG4OQD7sexwc1uV2DmWsXHQ","hostname":"www.projectbaiu.org","port":"80","addressesResolved":["63.226.21.50"],"addressUsed":"63.226.21.50"}]\n["validated"]\t"2022-04-28T17:12:46Z"']
• Cleaning challenge tokens...
• (hook) Cleaning Challenge
• (hook) Challenge rule removed from virtual.
• (hook) Cleaning Challenge
  Traceback (most recent call last):
  File "/shared/letsencrypt/hook_script.py", line 120, in
  clean_challenge(sys.argv[2:])
  File "/shared/letsencrypt/hook_script.py", line 62, in clean_challenge
  vip.properties['rules'].remove('/Common/le_challenge_rule')
  ValueError: list.remove(x): x not in list
  ERROR: clean_challenge hook returned with non-zero exit code

In writing a script to go through our 85 F5 instances to build an inventory with VS, pool, node, and SSL information returned, we find the iControlREST times out on occasion. Rather than have the whole script fail, we want it to continue processing as if no data was received.

I have edited big.py to encapsulate the "response = self.session.get(url, timeout=self.timeout) with a try statement, and if it fails to connect, the except sets response = ''

I have done this in the load and show definitions of big.py file. Hopefully this will be helpful to others vs writing all kinds of error handling into your main script.

Similar to the load method, the show method now also shows a single object if called with an object name.
It continues to return a list if called without an object name.

I could not use 'if "items" in response_json:' for stats because the API response is different then the when using the load method.
Had to use 'if "collection" in response_json["kind"]:'.

I have written a connection class using the following method:

def f5_conn(f5ip: str) -> BIGIP:
    """Function to create a connection to f5"""
    try:
        return BIGIP(device=f5ip, username=secrets["f5Username"], password=secrets["f5Password"], login_provider="tmos", request_token=False)

The above class gets invoked every time there is an API request to the F5 device.
We saw an error in our API that said:

"user xxxx-xxxx-xxxx-xxxx-xxxxxxxx has reached maximum active login tokens"

So, I decided to delete all tokens using a DELETE call to the /mgmt/shared/authz/tokens endpoint and then making sure request_token was set to False

Once I made a simple GET call to the device, I notice the device is still issuing tokens for that user.

Refactor the load method in bigip.py so that only collections return a list. This will allow the behaviors on load to be consistent:

# Collection
pools = br.load('/mgmt/tm/ltm/pool')
for p in pools:
    for k, v in iter(p.properties.items()):
        print(k, v)

# Named Object
pool = br.load('/mgmt/tm/ltm/pool/testpool')
for k, v in iter(pool.properties.items()):
    print(k, v)

Currently, you have to manipulate the pool on load or in the loop like this:

pool = br.load('/mgmt/tm/ltm/pool/testpool')[0]
for k, v in iter(pool.properties.items()):
    print(k, v)
## OR ##
pool = br.load('/mgmt/tm/ltm/pool/testpool')
for k, v in iter(pool[0].properties.items()):
...     print(k, v)

We would like to use this package but our security team won't allow it unless it is bundled into a release and has a direct download to a specific version. It should only take a couple of minutes to do: https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository

Hi @leonardobdes, can you update the package requirements to install requests if not already present on new installations?

BIGREST Version: 1.4.0
BIG-IP Version: 14.1.4.6 Build 0.0.8

To replicate bug, simply try to create an instance of the BIGIP class
bigip = BIGIP(hostname, username, password)

stack trace:

_connect, big.py:491
__init__, big.py:91
__init__, bigip.py:40

The issue is line 491. During _connect(), BIGIP tries to GET on f"https://{self.device}/mgmt/shared/echo-query"
In my version of BIG-IP, this URL does not exist. My response status code is 401, so BIGIP will raise RESTAPIError on line 493.

The solution is simple (at least for me). Just get rid of the mgmt/shared/echo-query from the URL. Use this URL instead:
f"https://{self.device}/

This returns a 200 status, so everything works.

My change is here:
https://github.com/rahvee/BIGREST/commit/f535bfaec341741f90bd0c90cd21f60ebad628a3

This change works for me. I don't have the ability to test on any other system (different versions of Big-IP, IQ), so I didn't submit it as a pull request for you.

I will be working on this one for a while. Goal is to use pytest to:

• test python version compatibility
• test python style compliance
• test BIG-IP version compatibility
• test BIG-IQ version compatibility

Hi, what does this actually provide? I know it provides some things that apparently make it easier to work with both BIGIP and BIGIQ but more examples are needed. The reason behind its creation.... to provide a unified REST API to BIG-IP products perhaps?

Hello - I tried to make a connection to our new BIG-IQ. I get the error below. admin user account has the following roles: Administrator Role, F5 Device Trust User and Device Manager.

>>> bq = BIGIQ("10.14.49.94","admin","xZxZxZxZx",session_verify=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/appviewx/apineda/venv/lib/python3.8/site-packages/bigrest/big.py", line 96, in __init__
    self._connect()
  File "/home/appviewx/apineda/venv/lib/python3.8/site-packages/bigrest/big.py", line 499, in _connect
    raise RESTAPIError(response, self.debug)
bigrest.common.exceptions.RESTAPIError: 
Status:
401
Response Body:
<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>webd</center>
</body>
</html>

BIGREST seems to consider only the 200 code as a successful request. Should be updated to include the other successful codes (201, 202, etc).

Example:

Traceback (most recent call last):
  File "XXXX.py", line 52, in <module>
    create = device.create(f'/mgmt/tm/asm/policies/{rest_format(policyId)}/whitelist-ips', whitelist_settings)
  File "/home/nick/.local/lib/python3.8/site-packages/bigrest/big.py", line 184, in create
    raise RESTAPIError(response, self.debug)
bigrest.common.exceptions.RESTAPIError: 
Status:
201
Response Body:
{
    "ignoreIpReputation": true,
    "blockRequests": "never",
    "ignoreAnomalies": false,
    "neverLogRequests": true,
 …
}

Add ability to see curl format of every request sent as a list item in a debug attribute, whether or not there is an error.

Requires no change to the debug='file' submission, but will take that and:

• Set debug attribute to True
• Set debug_file attribute to the filename
• Append all requests as curl formatted commands to debug_output list attribute
• Still allow for debug file to be created on errors