Seeing it on your PC
Normal while a built-in Windows VPN or a corporate VPN client is connecting.
Should you block it?
Allow it outgoing if you use one of these VPNs. A home PC has no reason to accept incoming 500 or 4500.
Check it on your PC
Run this in PowerShell to see what is using port 500 right now, and which program owns it:
Get-NetUDPEndpoint -LocalPort 500 -ErrorAction SilentlyContinue | Select-Object LocalAddress,LocalPort,@{n='Process';e={(Get-Process -Id $_.OwningProcess -ErrorAction SilentlyContinue).ProcessName}}No output means nothing is using the port at the moment. For every connection at once, paste netstat -ano into the netstat explainer.
Block it in Windows Firewall
In PowerShell run as administrator, this stops other computers from connecting to this port on your PC. It does not stop your own programs connecting out.
New-NetFirewallRule -DisplayName "Block inbound UDP 500,4500" -Direction Inbound -Protocol UDP -LocalPort 500,4500 -Action Block
Undo it with Remove-NetFirewallRule -DisplayName "Block inbound …", using the same name. If you depend on the service on your local network, this rule stops that too.