PfSense 23.01/2.7 (FreeBSD 14.0) about pfatt HOT 58 CLOSED

If I'm reading the comment correctly, I think someone solved this without using the script: https://redmine.pfsense.org/issues/12070#note-7

If someone can contact them and ask about the procedure?

I just tried what they suggested tonight and wasn't able to get it working, I posted in that thread in hopes that he will respond.

from pfatt.

So the good news is I was able to get it working with the old school script but it's still using netgraph. :(

Maybe someone with a little more skills and time can get it working without netgraph, I've used up all of my political capital with the family today :)

I will post the script below (see attached)

As you can see I only added the "/sbin/ifconfig $ONT_IF -vlanhwfilter" under the enabling promisc section for my supplicant option, if you use the bridge go ahead and do the same under the bridge section of the script. (should only be required for Intel NIC's)

This doesn't require any custom driver anymore so at least there is a small win there.
2301_bypass.txt

from pfatt.

FYI to those on this thread, all of the UI elements are added into pfSense Plus 23.05 to do this without a script and no netgraph required. You can enable promiscuous mode for the modem interface, set a PCP value for the WAN interface, and setup the 802.1X bridging using the new Ethernet Rules UI under Firewall --> Rules (Need to go to System --> Advanced --> Firewall & NAT --> Ethernet Filtering and enable the option first). I'm currently running an ATT connection bypassed with the bridge method (not wpa_supplicant) and it's been running for a few weeks trouble free.

If you are running an e1000-based NIC on either/both of your modem and WAN interfaces, you will need to disable vlanfwfilter still with an earlyshellcmd script (ifconfig igb# -vlanhwfilter) but any other NIC should just work.

Screenshot of the Ethernet Rules in Action:

from pfatt.

@TTK-199837 I just did this. I ended up reflashing to 23.05 as I could never get the upgrade to work correctly. I was using pfatt.sh tethered and then with supplicant on 23.01. On 23.05, I am using the tethered authbridge builtin to pfsense 23.05. Their manual outlines the steps perfectly - make sure you don't miss any steps. One thing I've noticed is that there is a long ~60s delay waiting for the WAN interface during the boot sequence. Not sure what thats all about but it seems like I read other people have it too.

from pfatt.

Well I'm posting this from a zero-netgraph supplicant setup after a reboot 😎

And as a side-effect my bufferbloat score has gone from B to A, without any traffic shaping (which I couldn't set up before anyway due to using netgraph) I think my B was from pfSense 22.05, can't remember if I tried on 23.01 before this

My approach was using wpa_cli's -a flag which lets you run a hook script every time the interface dis/connects. I use that to run ifconfig $ifname pcp 0 every time wpa_supplicant connects. I have no idea how well it handles disconnects though. I'll attach my current minimal scripts. I'm running this as a shellcmd at startup.

Edit: the wpa_cli hook does not handle disconnects well. When my fiber dropped out I had to reboot the switch to get IPv6 back. The disconnect hook never fired. That really makes me want to use dhcpcd hooks but I don't have time right now to figure all that out.

pfatt_sup14.txt
pfatt_sup14_action.txt

from pfatt.

Sure:

mvneta2: flags=28b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
	description: WAN
	options=800bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE>

so you have these I don't: TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP

and I have LINKSTATE

edit: I just discovered ifconfig -m $IFNAME which also displays the available capabilities. The only extra one I have that is disabled is LRO. I notice there's no VLAN_HWFILTER even as an option for mine...

capabilities=804bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,LINKSTATE>

from pfatt.

I have both supplicant and bridge/tethered bypass working with netgraph on pfSense 23.01 on Netgate hardware (Intel igb nics) with ipv6. I have a BGW210 gateway. The included Intel drivers work fine for this release but you do have to add the vlanhwfilter flags to ifconfig in both bridge and supplicant modes per Issue 65. I am using certificates from an old device with a different MAC than my assigned gateway. The only issue I've seen is pfSense struggles to update packages (super slow or fails) when ipv6 is enabled. Without ipv6 it is quick.

pfatt.txt

from pfatt.

Hello, looking for some help. I'm currently using pfsense 2.4.5 with pfatt in supplicant with working certs.

I purchased a DFP-34X-2C2 ONT stick, cloned the white ATT GPON ONT and installed in a pcie card in the pfsense server.

I'm trying to figure out what necessary changes to the pfatt script are required? Netgraph is still needed to handle the VLAN0 situation, but I also need the WAN VLAN Tagged on 962 because of where internet is available from the DFP ONT.

# omcicli mib get 84
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
VlanTagFilterData
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
=================================
EntityID: 0x1102
FilterTbl[0]: PRI 0,CFI 0, VID 962
FwdOp:  0x02
NumOfEntries: 1
=================================
=================================
EntityID: 0x110b
FilterTbl[0]: PRI 0,CFI 0, VID 962
FwdOp:  0x10
NumOfEntries: 1
=================================

Please open up another thread to discuss this

from pfatt.

How can I check for that? I remember it breaking the last two times I updated, so I am being very caution this time. Don't need the household yelling at me for another multi hour downtime haha. Thanks for the help!

If you have an Intel igb or em based NIC and had to load the custom driver last update then you will have an issue as those were compiled with the previous version of PfSense which was running on an older version of FreeBSD 12.3.
23.01 is running on FreeBSD 14 so you can't use that driver, you can however just disable hwvlanfilter and use the native driver and pfatt script (feel free to see mine above and compare to yours to see what I changed) and then everything should work fine.
This is probably the biggest change in a long time with them moving two major revisions of FreeBSD ahead and without me knowing your setup I can only speak to pfatt and nothing else so make sure you do your research and prepare for the worst and hope for the best.
GL

Hi again! Requesting some more hand holding if you do not mind ;)
You mention "just disable hwvlanfilter", however I am not familiar with this setting, or having ever enabled it. After some exploring, I believe these steps should remove / disable the old custom intel drivers, and have a working router, please let me know if I am correct:

• Reverse steps found here: Intel igb/em Interfaces Broken on 2.6/22.01+ #67 (comment)
• Remove old intel driver if_igb.ko from /boot/kernel
• Remove if_igb_load="YES" and if_igb_name="/boot/modules/if_igb.ko" from /boot/loader.conf
• (?) should I keep my current pfatt.sh unchanged? It seems a LOT different than the couple posted here, I believe because I am using bridge mode? Do I, or should I modify it in anyway? (https://github.com/TTK-199837/pfatt/blob/master/bin/pfatt.sh)
• Update via GUI, and we are up and running?

Thanks for your time!

You should check out the script I posted at the very top of this thread (3rd post) https://github.com/MonkWho/pfatt/files/10418690/2301_bypass.txt

I made this edit for supplicant mode (bolded part is my edit) /usr/bin/logger -st "pfatt" "enabling promisc for $ONT_IF..." /sbin/ifconfig $ONT_IF up /sbin/ifconfig $ONT_IF promisc /sbin/ifconfig $ONT_IF -vlanhwfilter

To do the same for bridge mode just add this part to bridge section /usr/bin/logger -st "pfatt" "enabling promiscuous mode..." /sbin/ifconfig $EAP_BRIDGE_IF promisc /sbin/ifconfig $ONT_IF promisc /sbin/ifconfig $ONT_IF -vlanhwfilter

BTW you are correct on how to remove all of the previous custom driver stuff from the previous version.

Also if you still want to use your script add something like this to the end

echo -n "$(getTimestamp) enabling promiscuous mode on $ONT_IF... " /sbin/ifconfig $ONT_IF promisc /sbin/ifconfig $ONT_IF -vlanhwfilter echo "OK!"

Or you can do a whole new section if you want

echo -n "$(getTimestamp) disabling vlanhwfilter on $ONT_IF... " /sbin/ifconfig $ONT_IF -vlanhwfilter echo "OK!"

Hope this helps

You are a lifesaver. Thank you very much! Update went smoothly and as planned!

from pfatt.

@bigjohns97 thanks for posting how you got it working and a copy of your script. Got supplicant working for the first time after 1 year on bridged, and now I'm on 23.01 to boot. Thanks again!

from pfatt.

After playing around with settings I got it to work. A few notes/steps below may help others. Referring back to what @ChronicledMonocle mentioned a couple of posts above.

* Upgrade to 23.05-RC to get the option for Ethernet Filtering

* Attached a USB NIC for my connection to the modem (if you have a free port on your pfSense you can use that)

* Added the new USB interface via Interfaces -> Assignments

* The only setting I changed on the new USB NIC interface for the Modem is to enable "Promiscuous Mode"; left the IPv4 and IPv6 Configuration types as None

* I couldn't find the "PCP" setting on the WAN interface (I guess it's called Priority Tag?), I left it blank; just ensured that the MAC Address field is set to the MAC of the AT&T Modem

* Added the two rules mentioned by @ChronicledMonocle  in the screenshot under Firewall -> Rules -> Ethernet

* Need to use the Advanced Options and set the Bridge To field; and the first rule needs to have the Protocol set to IEEE 802.1X

This worked for me without any restarts (except when upgrading pfSense to 23.05-RC). I haven't tried to see if this will survive a reboot of the modem and the pfSense router yet.

We have a documentation recipe going up on docs.netgate.com that will have a configuration example of how to do this step by step. I wrote this with input from the pfSense Plus TAC and Development teams for editing and feedback. Attached is a PDF of the draft before it goes live there. Bear in mind you need to be on pfSense Plus 23.05 RC for these instructions to work:
Configuring WAN Connectivity for 802.1X Authentication Bridging and VLAN0 PCP Tagging.pdf

from pfatt.

Also, the e1000 bug with vlanhwfilter should be fixed in the latest 23.05 RC release, so should be no need to disable vlanhwfilter anymore on those NICs.

from pfatt.

One thing I've noticed is that there is a long ~60s delay waiting for the WAN interface during the boot sequence. Not sure what thats all about but it seems like I read other people have it too.

Ahh... That dreaded 60s pause at "configuring wan interface..."

That happens specifically because eapol auth hasn't taken place yet and dhcp can't pull an IP. We ran into the same issue with wpa_supplicant method. The work around was to start the supplicant daemon as earlyshellcmd. Then once past the configuring wan.... something happens to the wan interface leaving wpa_supplicant in an unresponsive (to eapol requests) state. So the solution was to have wpa_supplicant launched twice, once as earlyshellcmd, once as shellcmd. The latter killing the existing process first.

Since you can't quite do that with the bridged method, your best best may be to change the wan dhcp timeout (under advanced wan settings) to something like 10-15s instead of the default 60s.

from pfatt.

Some extra threads for reference

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224961
https://www.freebsd.org/cgi/man.cgi?query=ifconfig&apropos=0&sektion=8&manpath=FreeBSD+14.0-CURRENT&arch=default&format=html

I tried just about everything to get this thing to listen, you do have to remove the previously compiled drivers from the other thread or the system halts, also you can't run any pfatt scripts because the ngeth0 interface comes up with a random MAC, and either way it doesn't seem like it would work based upon the above threads.

from pfatt.

If I'm reading the comment correctly, I think someone solved this without using the script:
https://redmine.pfsense.org/issues/12070#note-7

If someone can contact them and ask about the procedure?

from pfatt.

Is there any known fix for this?

from pfatt.

While trying out 23.01 RC, I found another way to get ATT working by using same VPID for WAN and OPT and keep pfSense and ATT RG as dual gateway, confirmed on 22.05 without pfatt. hope if helps:
https://navigateheadwinds.blogspot.com/2023/02/pfsense-att-dual-gateway.html

from pfatt.

Is there any known fix for this?

Known fix for what?

I posted the edit of the original script to use with Intel NIC.

Still working on trying to figure out how to not require the full script and take ngeth out of the equation.

from pfatt.

Sorry, but could someone clarify for me? If I have pfatt working fine on 22.05 will upgrading to 23.01 break it?

from pfatt.

It depends are you using the custom compiled driver?

from pfatt.

How can I check for that? I remember it breaking the last two times I updated, so I am being very caution this time. Don't need the household yelling at me for another multi hour downtime haha. Thanks for the help!

from pfatt.

How can I check for that? I remember it breaking the last two times I updated, so I am being very caution this time. Don't need the household yelling at me for another multi hour downtime haha. Thanks for the help!

If you have an Intel igb or em based NIC and had to load the custom driver last update then you will have an issue as those were compiled with the previous version of PfSense which was running on an older version of FreeBSD 12.3.

23.01 is running on FreeBSD 14 so you can't use that driver, you can however just disable hwvlanfilter and use the native driver and pfatt script (feel free to see mine above and compare to yours to see what I changed) and then everything should work fine.

This is probably the biggest change in a long time with them moving two major revisions of FreeBSD ahead and without me knowing your setup I can only speak to pfatt and nothing else so make sure you do your research and prepare for the worst and hope for the best.

GL

from pfatt.

from pfatt.

@bigjohns97 are you using wpa-supplicant/is that relevant in this issue? I've been trying to navigate this whole thing across different threads/bugs and I'm thinking maybe there's a mixture of feedback between wpa-supplicant and passthrough setups out there. FWIW I have a supplicant setup and also would like to get rid of netgraph. Even with netgraph I've always needed wpa-supplicant running on the "raw" interface so that its traffic doesn't get VLAN tags. I've been trying to come up with some clever setup with epair and a bridge or something like that but can never get the return traffic routed to the right interface.

from pfatt.

@bigjohns97 are you using wpa-supplicant/is that relevant in this issue? I've been trying to navigate this whole thing across different threads/bugs and I'm thinking maybe there's a mixture of feedback between wpa-supplicant and passthrough setups out there. FWIW I have a supplicant setup and also would like to get rid of netgraph. Even with netgraph I've always needed wpa-supplicant running on the "raw" interface so that its traffic doesn't get VLAN tags. I've been trying to come up with some clever setup with epair and a bridge or something like that but can never get the return traffic routed to the right interface.

Yes I am using supplicant.

However, in my testing with this I never did the wpa_supplicant before applying the PCP values, I can try this later on today and see if it makes a difference.

from pfatt.

Well I'm posting this from a zero-netgraph supplicant setup after a reboot 😎

And as a side-effect my bufferbloat score has gone from B to A, without any traffic shaping (which I couldn't set up before anyway due to using netgraph) I think my B was from pfSense 22.05, can't remember if I tried on 23.01 before this

My approach was using wpa_cli's -a flag which lets you run a hook script every time the interface dis/connects. I use that to run ifconfig $ifname pcp 0 every time wpa_supplicant connects. I have no idea how well it handles disconnects though. I'll attach my current minimal scripts. I'm running this as a shellcmd at startup.

pfatt_sup14.txt pfatt_sup14_action.txt

I wasn't able to get mine to authenticate using these scripts even tried -vlanhwfilter as an extra step to see if it was that.

No-Go

from pfatt.

I wasn't able to get mine to authenticate using these scripts even tried -vlanhwfilter as an extra step to see if it was that.

Ugh sorry. FWIW I'm using a Netgate SG-3100 which is 32-bit ARM and Marvell network interfaces. I can provide command or log output of anything that would help

from pfatt.

On reboot number 2, wpa_supplicant logged a failed auth, then tried again 30 seconds later and succeeded

from pfatt.

Are you running this as a earlyshellcmd?

That is how I was trying to use it and once booted up I could run ifconfig and noticed that I didn't have pcp assigned to the interface.

I am wondering if you could post how you came up with the order of things.

Judging by your script.

1. bring up interface and set promisc mode
2. remove any previous pcp attribute
3. run wpa auth along with connected watcher (4,5 below)
4. if not connected add pcp attribute
5. once connected remove pcp attribute

The whole time wpa and connected watcher stays running in the background.

Is this accurate?

from pfatt.

BTW I noticed you reference en interface so your WAN is intel en?

from pfatt.

Tried again this morning, toggling everything I could

with and without pcp
with and without vlanhwfilter
using my RG MAC as the wpa identity
using the purchased certs wpa identity

I could never get the wpa process to successfully authenticate, and I waited until it told me it failed.

I am wondering if this has anything to do with my certs being associated to another MAC than my RG, I still assign my RG MAC to my ngeth and my igb0 interface regardless but it just seems to work when using ngeth.

At a dead end, up for trying anything at this point if anyone has any ideas.

from pfatt.

BTW I noticed you reference en interface so your WAN is intel en?

No sorry that was pretty bad on my part, en123 is a placeholder name, and it's a Linuxism at that, where every wired iface gets named en<something>. My real WAN is mvneta2

Are you running this as a earlyshellcmd?

Only tried as a shellcmd so far. I want to try as an earlyshellcmd soon.

I am wondering if you could post how you came up with the order of things.

Judging by your script.

1. bring up interface and set promisc mode

2. remove any previous pcp attribute

3. run wpa auth along with connected watcher (4,5 below)

4. if not connected add pcp attribute

5. once connected remove pcp attribute

The whole time wpa and connected watcher stays running in the background.

Is this accurate?

Yep that's all accurate. wpa_cli is a second long-running process alongside wpa_supplicant. I was looking for any of the standard daemons that might support external hooks. And actually I think long-term doing this as dhcpcd hooks instead of supplicant hooks might be a little more reliable, but that's just a hunch, and idek if you can add config options like that to the dhcpcd instances that pfSense automatically runs for interfaces.

To get here I'd been experimenting with setting my WAN interface in pfSense to the real WAN iface (mvneta2 in my case) instead of ngeth0. Through a combo of watching the system logs in the web UI and doing some packet captures I saw for whatever reasons in my setup, my ONT will only respond to 802.1X frames that do NOT have any 802.1Q tags, and will only respond to regular traffic that DOES have 802.1Q tags. I toggled the tags by running ifconfig mvneta2 pcp 0 or -pcp and occasionally dis/enabling the interface. I was struggling for a long time to get a non-netgraph equivalent of two network interfaces that had the same external MAC, so I could set pcp 0 on one and leave wpa_supplicant on the other. Using connection hooks to toggle it instead on one interface was a shower thought and only worked because these daemons happen to support hooks.

I don't know if promisc mode is still needed. That's left from original pfatt.

For debugging this script live I was ssh'd in and alternating between running this script and running a pfatt_cleanup.txt script I threw together. All the while watching the system logs in web UI for output from supplicant/etc.

I am wondering if this has anything to do with my certs being associated to another MAC than my RG, I still assign my RG MAC to my ngeth and my igb0 interface regardless but it just seems to work when using ngeth.

My RG MAC is forced on mvneta2 in the web UI. Can't help with the different cert identity unfortunately.

If you haven't already I'd recommend getting a packet capture in the web UI of a successful supplicant handshake with your known good setup, download it and check it out in Wireshark or something. This probably means booting normally, physically disconnecting the cable, starting capture, and then reconnecting the cable. That should at least give a baseline of if it's expecting .Q tags or not.

from pfatt.

Well unfortunately the packet captures didn't produce any valuable information.

The EAPOL start frame looks exactly the same on your script and the original pfatt script.

The only difference is the response which through the pfatt script is a success and your script is replies on VLAN 0 requesting an identity.

I'm going to have to wait for someone with more skills than myself to come up with a solution.

from pfatt.

from pfatt.

When I was running ESXi I would just create the NIC with my RG MAC at the virtual layer and tag VLAN 0 at the virtual layer as well and everything would work fine.

When you do an ifconfig, do you see an ether address as well as a hwaddr? Are they the same or different? Are they your RG MAC?

from pfatt.

The MACs are different. ether is my RG MAC, but hwaddr is 00:08:a2:something (same prefix for all my real interfaces)

from pfatt.

The ngeth interface had a different hwaddr as well. I guess I need to dig into that and really see what makes that different than what I am doing.

from pfatt.

When I was running ESXi I would just create the NIC with my RG MAC at the virtual layer and tag VLAN 0 at the virtual layer as well and everything would work fine.

I find this especially weird...is there some standards-edge-case about eapol starts never supposed to be tagged? Otherwise if ESXi was tagging the start packet (which I would've assumed, just out of a hope for sanity) then why was the ONT accepting those starts on VLAN 0 but not accepting on pfSense if you just put supplicant on an interface with pcp 0 all the time?

The ngeth interface had a different hwaddr as well. I guess I need to dig into that and really see what makes that different than what I am doing.

Let me paste my supplicant conf too. I do have a cert that matches my provisioned RG MAC, so that is different, but I can't imagine that affecting this .Q stuff.

eapol_version=2
ap_scan=0
fast_reauth=1
network={
        ca_cert="/conf/pfatt/wpa/ca.pem"
        client_cert="/conf/pfatt/wpa/client.pem"
        eap=TLS
        eapol_flags=0
        identity="[redacted RG MAC]"
        key_mgmt=IEEE8021X
        phase1="allow_canned_success=1"
        private_key="/conf/pfatt/wpa/private.pem"
}

I guess you could also try adding a line into the script that forces the RG MAC on the interface before it comes up. I'm relying on pfSense config to do that, and I could imagine that working for shellcmd but not earlyshellcmd, if the script happens before pfSense applies all the user config options.

from pfatt.

I tried the line to the script setting the ether addr but one thing I just noticed is your eapol_version is 2 and mine is 1.

Checking the original pfatt script it looks like it has it set to 2 as well, will check to see if this makes a difference a little later on tonight.

from pfatt.

Still a negative, even trued using my identity MAC as my spoofed MAC on my WAN interface this time thinking maybe that would work.

no go

Also tried setting a pcp of 7 since that is what the ONT was replying with when asking for my identity, still didn't work.

Giving up, surely someone else will come along and test out your script and see if it's me or maybe you just have something special with those realtek interfaces.

from pfatt.

@unawarez would you mind posting your interface options?

Here are mine
RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP
for a while we had to disable vlanhwfilter for the pfatt script to work whenever they moved to a consolidated igb and em intel driver. (I still do this BTW and have been trying this in my testing, just wondering if there are more I should be disabling)

I am wondering if one of these are disabled on your driver by default and may explain our different experiences.

from pfatt.

Hello, looking for some help. I'm currently using pfsense 2.4.5 with pfatt in supplicant with working certs.

I purchased a DFP-34X-2C2 ONT stick, cloned the white ATT GPON ONT and installed in a pcie card in the pfsense server.

I'm trying to figure out what necessary changes to the pfatt script are required?
Netgraph is still needed to handle the VLAN0 situation, but I also need the WAN VLAN Tagged on 962 because of where internet is available from the DFP ONT.

# omcicli mib get 84
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
VlanTagFilterData
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
=================================
EntityID: 0x1102
FilterTbl[0]: PRI 0,CFI 0, VID 962
FwdOp:  0x02
NumOfEntries: 1
=================================
=================================
EntityID: 0x110b
FilterTbl[0]: PRI 0,CFI 0, VID 962
FwdOp:  0x10
NumOfEntries: 1
=================================

from pfatt.

How can I check for that? I remember it breaking the last two times I updated, so I am being very caution this time. Don't need the household yelling at me for another multi hour downtime haha. Thanks for the help!

If you have an Intel igb or em based NIC and had to load the custom driver last update then you will have an issue as those were compiled with the previous version of PfSense which was running on an older version of FreeBSD 12.3.

23.01 is running on FreeBSD 14 so you can't use that driver, you can however just disable hwvlanfilter and use the native driver and pfatt script (feel free to see mine above and compare to yours to see what I changed) and then everything should work fine.

This is probably the biggest change in a long time with them moving two major revisions of FreeBSD ahead and without me knowing your setup I can only speak to pfatt and nothing else so make sure you do your research and prepare for the worst and hope for the best.

GL

Hi again! Requesting some more hand holding if you do not mind ;)

You mention "just disable hwvlanfilter", however I am not familiar with this setting, or having ever enabled it. After some exploring, I believe these steps should remove / disable the old custom intel drivers, and have a working router, please let me know if I am correct:

• Reverse steps found here: #67 (comment)

• Remove old intel driver if_igb.ko from /boot/kernel

• Remove if_igb_load="YES" and if_igb_name="/boot/modules/if_igb.ko" from /boot/loader.conf

• (?) should I keep my current pfatt.sh unchanged? It seems a LOT different than the couple posted here, I believe because I am using bridge mode? Do I, or should I modify it in anyway? (https://github.com/TTK-199837/pfatt/blob/master/bin/pfatt.sh)

• Update via GUI, and we are up and running?

Thanks for your time!

from pfatt.

@bigjohns97 Did you ever get the igb nonsense figured out?

I have a i211 passed through using proxmox. Get absolutely no eapol traffic on tcpdump in 23.01. I can have proxmox handle the vlan0 tagging which results in virtio drivers for wan. This works, but at line speeds, cpu usage is way high (top in proxmox).

In sophos utm, with the i211 passsed, i'm seeing roughly 1/4 of the cpu usage doing line speed upload/download tests. Would like to move to pf, but this is not acceptable.

from pfatt.

How can I check for that? I remember it breaking the last two times I updated, so I am being very caution this time. Don't need the household yelling at me for another multi hour downtime haha. Thanks for the help!

If you have an Intel igb or em based NIC and had to load the custom driver last update then you will have an issue as those were compiled with the previous version of PfSense which was running on an older version of FreeBSD 12.3.
23.01 is running on FreeBSD 14 so you can't use that driver, you can however just disable hwvlanfilter and use the native driver and pfatt script (feel free to see mine above and compare to yours to see what I changed) and then everything should work fine.
This is probably the biggest change in a long time with them moving two major revisions of FreeBSD ahead and without me knowing your setup I can only speak to pfatt and nothing else so make sure you do your research and prepare for the worst and hope for the best.
GL

Hi again! Requesting some more hand holding if you do not mind ;)

You mention "just disable hwvlanfilter", however I am not familiar with this setting, or having ever enabled it. After some exploring, I believe these steps should remove / disable the old custom intel drivers, and have a working router, please let me know if I am correct:

• Reverse steps found here: Intel igb/em Interfaces Broken on 2.6/22.01+ #67 (comment)
• Remove old intel driver if_igb.ko from /boot/kernel
• Remove if_igb_load="YES" and if_igb_name="/boot/modules/if_igb.ko" from /boot/loader.conf
• (?) should I keep my current pfatt.sh unchanged? It seems a LOT different than the couple posted here, I believe because I am using bridge mode? Do I, or should I modify it in anyway? (https://github.com/TTK-199837/pfatt/blob/master/bin/pfatt.sh)
• Update via GUI, and we are up and running?

Thanks for your time!

You should check out the script I posted at the very top of this thread (3rd post) https://github.com/MonkWho/pfatt/files/10418690/2301_bypass.txt

I made this edit for supplicant mode (bolded part is my edit)
/usr/bin/logger -st "pfatt" "enabling promisc for $ONT_IF..."
/sbin/ifconfig $ONT_IF up
/sbin/ifconfig $ONT_IF promisc
/sbin/ifconfig $ONT_IF -vlanhwfilter

To do the same for bridge mode just add this part to bridge section
/usr/bin/logger -st "pfatt" "enabling promiscuous mode..."
/sbin/ifconfig $EAP_BRIDGE_IF promisc
/sbin/ifconfig $ONT_IF promisc
/sbin/ifconfig $ONT_IF -vlanhwfilter

BTW you are correct on how to remove all of the previous custom driver stuff from the previous version.

Also if you still want to use your script add something like this to the end

echo -n "$(getTimestamp) enabling promiscuous mode on $ONT_IF... "
/sbin/ifconfig $ONT_IF promisc
/sbin/ifconfig $ONT_IF -vlanhwfilter
echo "OK!"

Or you can do a whole new section if you want

echo -n "$(getTimestamp) disabling vlanhwfilter on $ONT_IF... "
/sbin/ifconfig $ONT_IF -vlanhwfilter
echo "OK!"

Hope this helps

from pfatt.

@bigjohns97 Did you ever get the igb nonsense figured out?

I have a i211 passed through using proxmox. Get absolutely no eapol traffic on tcpdump in 23.01. I can have proxmox handle the vlan0 tagging which results in virtio drivers for wan. This works, but at line speeds, cpu usage is way high (top in proxmox).

In sophos utm, with the i211 passsed, i'm seeing roughly 1/4 of the cpu usage doing line speed upload/download tests. Would like to move to pf, but this is not acceptable.

If you want try disabling hardware checksum offload in pfsense on the guest and let me know if that solves your CPU usage issues.

(all three of these should be checked)

As to me getting where I would like to be I still haven't been able to get a good understanding why we can't remove netgraph and get it to work, I did get this message from ccope

Which would be great if he shared but I didn't want to push anymore than I already have.

from pfatt.

@bigjohns97 Unchecking all 3 of those leads to a slight marginal improvement in cpu usage for virtio. How ever the webgui becomes painfully sluggish.

By default the bottom 2 options are check marked.

Can you clarify what is the state of eapol traffic with igb (i211) driver/nic? Should this be working somehow?

from pfatt.

Wanted to update. Yes, the netgraph method still works in 23.01 with those ifconfig switches.

What's unclear; Is there still a need for netgraph?

With a i211, I could not get it to process any eapol traffic without netgraph.

from pfatt.

Want to point out that @bigjohns97's script "2301_bypass.txt" doesn't have the -vlanhwfilter on the bridge mode only on the supplicant mode. Both modes need the -vlanhwfilter on the ifconfig $ONT_IF promisc command.

from pfatt.

@ChronicledMonocle Great news! Thanks for sharing. Is there a link to step-by-step instructions on setting this up via UI?

from pfatt.

After playing around with settings I got it to work. A few notes/steps below may help others. Referring back to what @ChronicledMonocle mentioned a couple of posts above.

• Upgrade to 23.05-RC to get the option for Ethernet Filtering
• Attached a USB NIC for my connection to the modem (if you have a free port on your pfSense you can use that)
• Added the new USB interface via Interfaces -> Assignments
• The only setting I changed on the new USB NIC interface for the Modem is to enable "Promiscuous Mode"; left the IPv4 and IPv6 Configuration types as None
• I couldn't find the "PCP" setting on the WAN interface (I guess it's called Priority Tag?), I left it blank; just ensured that the MAC Address field is set to the MAC of the AT&T Modem
• Added the two rules mentioned by @ChronicledMonocle in the screenshot under Firewall -> Rules -> Ethernet
• Need to use the Advanced Options and set the Bridge To field; and the first rule needs to have the Protocol set to IEEE 802.1X

This worked for me without any restarts (except when upgrading pfSense to 23.05-RC). I tested a restart of the pfSense box and the modem, and this setup survived the restart.

from pfatt.

After playing around with settings I got it to work. A few notes/steps below may help others. Referring back to what @ChronicledMonocle mentioned a couple of posts above.

* Upgrade to 23.05-RC to get the option for Ethernet Filtering

* Attached a USB NIC for my connection to the modem (if you have a free port on your pfSense you can use that)

* Added the new USB interface via Interfaces -> Assignments

* The only setting I changed on the new USB NIC interface for the Modem is to enable "Promiscuous Mode"; left the IPv4 and IPv6 Configuration types as None

* I couldn't find the "PCP" setting on the WAN interface (I guess it's called Priority Tag?), I left it blank; just ensured that the MAC Address field is set to the MAC of the AT&T Modem

* Added the two rules mentioned by @ChronicledMonocle  in the screenshot under Firewall -> Rules -> Ethernet

* Need to use the Advanced Options and set the Bridge To field; and the first rule needs to have the Protocol set to IEEE 802.1X

This worked for me without any restarts (except when upgrading pfSense to 23.05-RC). I haven't tried to see if this will survive a reboot of the modem and the pfSense router yet.

We have a documentation recipe going up on docs.netgate.com that will have a configuration example of how to do this step by step. I wrote this with input from the pfSense Plus TAC and Development teams for editing and feedback. Attached is a PDF of the draft before it goes live there. Bear in mind you need to be on pfSense Plus 23.05 RC for these instructions to work: Configuring WAN Connectivity for 802.1X Authentication Bridging and VLAN0 PCP Tagging.pdf

Good stuff. I would also recommend adding DHCPV6 Server and RA basic/recommended settings to the document.

from pfatt.

After playing around with settings I got it to work. A few notes/steps below may help others. Referring back to what @ChronicledMonocle mentioned a couple of posts above.

* Upgrade to 23.05-RC to get the option for Ethernet Filtering

* Attached a USB NIC for my connection to the modem (if you have a free port on your pfSense you can use that)

* Added the new USB interface via Interfaces -> Assignments

* The only setting I changed on the new USB NIC interface for the Modem is to enable "Promiscuous Mode"; left the IPv4 and IPv6 Configuration types as None

* I couldn't find the "PCP" setting on the WAN interface (I guess it's called Priority Tag?), I left it blank; just ensured that the MAC Address field is set to the MAC of the AT&T Modem

* Added the two rules mentioned by @ChronicledMonocle  in the screenshot under Firewall -> Rules -> Ethernet

* Need to use the Advanced Options and set the Bridge To field; and the first rule needs to have the Protocol set to IEEE 802.1X

This worked for me without any restarts (except when upgrading pfSense to 23.05-RC). I haven't tried to see if this will survive a reboot of the modem and the pfSense router yet.

We have a documentation recipe going up on docs.netgate.com that will have a configuration example of how to do this step by step. I wrote this with input from the pfSense Plus TAC and Development teams for editing and feedback. Attached is a PDF of the draft before it goes live there. Bear in mind you need to be on pfSense Plus 23.05 RC for these instructions to work: Configuring WAN Connectivity for 802.1X Authentication Bridging and VLAN0 PCP Tagging.pdf

I really think this pdf document is great and the write up on docs.netgate.com is even better but I still can’t get this setup working with my SG-5100 I think it may have something to do with the nic’s and not liking the new Ethernet firewall rules, but I get a DHCP address but I’m unable to route traffic. Not sure if anyone else is having this issue or not.

from pfatt.

Going to close out this issue as there is a newer version of 23.05 which when included with the new wpa_supplicant binary not only eliminates the need for Netgraph but also the need for stripper switches and even PCP (vlan0) values!

from pfatt.

After playing around with settings I got it to work. A few notes/steps below may help others. Referring back to what @ChronicledMonocle mentioned a couple of posts above.

* Upgrade to 23.05-RC to get the option for Ethernet Filtering

* Attached a USB NIC for my connection to the modem (if you have a free port on your pfSense you can use that)

* Added the new USB interface via Interfaces -> Assignments

* The only setting I changed on the new USB NIC interface for the Modem is to enable "Promiscuous Mode"; left the IPv4 and IPv6 Configuration types as None

* I couldn't find the "PCP" setting on the WAN interface (I guess it's called Priority Tag?), I left it blank; just ensured that the MAC Address field is set to the MAC of the AT&T Modem

* Added the two rules mentioned by @ChronicledMonocle  in the screenshot under Firewall -> Rules -> Ethernet

* Need to use the Advanced Options and set the Bridge To field; and the first rule needs to have the Protocol set to IEEE 802.1X

This worked for me without any restarts (except when upgrading pfSense to 23.05-RC). I haven't tried to see if this will survive a reboot of the modem and the pfSense router yet.

We have a documentation recipe going up on docs.netgate.com that will have a configuration example of how to do this step by step. I wrote this with input from the pfSense Plus TAC and Development teams for editing and feedback. Attached is a PDF of the draft before it goes live there. Bear in mind you need to be on pfSense Plus 23.05 RC for these instructions to work: Configuring WAN Connectivity for 802.1X Authentication Bridging and VLAN0 PCP Tagging.pdf

I really think this pdf document is great and the write up on docs.netgate.com is even better but I still can’t get this setup working with my SG-5100 I think it may have something to do with the nic’s and not liking the new Ethernet firewall rules, but I get a DHCP address but I’m unable to route traffic. Not sure if anyone else is having this issue or not.

Hi!

Were you able to get things working for your SG-5100?

from pfatt.

And a question for the experts (@bigjohns97) if I update pfsense via the GUI and keep all the current pfatt bypass files and earlyshellcmd scrips intact, will something break? Or will everything work like now, and give me the opportunity to switch over to the new method?

My current plan is to:

• Update via GUI and Reboot
• Remove the earlyshellcmd for /root/bin/pfatt.sh
• Follow https://docs.netgate.com/pfsense/en/latest/recipes/authbridge.html
• Reboot and pray

Is this the correct way of doing things?

Thank you!

from pfatt.

And a question for the experts (@bigjohns97) if I update pfsense via the GUI and keep all the current pfatt bypass files and earlyshellcmd scrips intact, will something break? Or will everything work like now, and give me the opportunity to switch over to the new method?

Unfortunately I can't answer this as I wasn't running the ngeth based script once 23.05 dropped, I was using a stripper switch by then.

from pfatt.

Wait so are you guys still tethered to the RG like the docs mention? One nic to ONT the other to the RG? https://docs.netgate.com/pfsense/en/latest/recipes/authbridge.html

Is it possible to use extracted certs with wpa_supplicant? I see no mention of it in the docs.
Was hoping to upgrade from 2.6 and continue to use my setup without having to pull the router out of the closet.

from pfatt.