allow zscaler

We’ll need a way to locate the potential traffic in logs to begin investigation. Is there a way to identify zscalar traffic? Does it come from a specific IP or IP range? You can email help@drupal.org if you do not want to disclose IPs on this public issue.

Does GIT_SSH_COMMAND="ssh -vvv" git clone git@git.drupal.org:project/dkan_dataset_archiver.git offer any help on what part is hanging?

Thanks for looking into this @drumm
I am not exactly sure how zscaler could be detected on your side. I investigate that some more.

Using the debug test with Zscaler on I get the following

08:36:55 » GIT_SSH_COMMAND="ssh -vvv" git clone git@git.drupal.org:project/dkan_dataset_archiver.git
Cloning into 'dkan_dataset_archiver'...
OpenSSH_9.9p2, LibreSSL 3.3.6
debug1: Reading configuration data /Users/steve.wirt/.ssh/config
debug1: /Users/steve.wirt/.ssh/config line 5: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/steve.wirt/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/steve.wirt/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug1: Connecting to git.drupal.org port 22.
debug1: Connection established.
debug1: identity file /Users/steve.wirt/.ssh/id_rsa type 0
debug1: identity file /Users/steve.wirt/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug3: fd 5 is O_NONBLOCK
debug1: Authenticating to git.drupal.org:22 as 'git'
debug3: record_hostkey: found key type ECDSA in file /Users/steve.wirt/.ssh/known_hosts:50
debug3: load_hostkeys_file: loaded 1 keys from git.drupal.org
debug1: load_hostkeys: fopen /Users/steve.wirt/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent

### Pauses here for over a minute, the proceeds

ssh_dispatch_run_fatal: Connection to 44.230.112.158 port 22: Operation timed out
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

With zscaler off I get

GIT_SSH_COMMAND="ssh -vvv" git clone git@git.drupal.org:project/dkan_dataset_archiver.git
Cloning into 'dkan_dataset_archiver'...
OpenSSH_9.9p2, LibreSSL 3.3.6
debug1: Reading configuration data /Users/steve.wirt/.ssh/config
debug1: /Users/steve.wirt/.ssh/config line 5: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/steve.wirt/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/steve.wirt/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug1: Connecting to git.drupal.org port 22.
debug1: Connection established.
debug1: identity file /Users/steve.wirt/.ssh/id_rsa type 0
debug1: identity file /Users/steve.wirt/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug3: fd 5 is O_NONBLOCK
debug1: Authenticating to git.drupal.org:22 as 'git'
debug3: record_hostkey: found key type ECDSA in file /Users/steve.wirt/.ssh/known_hosts:50
debug3: load_hostkeys_file: loaded 1 keys from git.drupal.org
debug1: load_hostkeys: fopen /Users/steve.wirt/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: sntrup761x25519-sha512,sntrup761x25519-sha512@openssh.com,mlkem768x25519-sha256,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,kex-strict-c-v00@openssh.com
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,kex-strict-s-v00@openssh.com
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: ciphers stoc: aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes256-ctr,aes256-cbc,aes128-gcm@openssh.com,aes128-ctr,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha1,umac-128@openssh.com,hmac-sha2-512
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug3: kex_choose_conf: will use strict KEX ordering
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:FeJiekFTTnXWc1GDUfFVcRckyZkZfdmks1dSqtS6OVg
debug3: record_hostkey: found key type ECDSA in file /Users/steve.wirt/.ssh/known_hosts:50
debug3: load_hostkeys_file: loaded 1 keys from git.drupal.org
debug1: load_hostkeys: fopen /Users/steve.wirt/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'git.drupal.org' is known and matches the ECDSA host key.
debug1: Found key in /Users/steve.wirt/.ssh/known_hosts:50
debug3: send packet: type 21
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug2: KEX algorithms: sntrup761x25519-sha512,sntrup761x25519-sha512@openssh.com,mlkem768x25519-sha256,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,kex-strict-c-v00@openssh.com
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp256,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug3: kex_input_ext_info: extension server-sig-algs
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug3: ssh_get_authentication_socket_path: path '/private/tmp/com.apple.launchd.RxKcTrXsvY/Listeners'
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 1 keys
debug1: Will attempt key: /Users/steve.wirt/.ssh/id_rsa RSA SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss explicit agent
debug2: pubkey_prepare: done
debug1: Offering public key: /Users/steve.wirt/.ssh/id_rsa RSA SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss explicit agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /Users/steve.wirt/.ssh/id_rsa RSA SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss explicit agent
debug3: sign_and_send_pubkey: using publickey with RSA SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss
debug3: sign_and_send_pubkey: signing using rsa-sha2-512 SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to git.drupal.org ([44.230.112.158]:22) using "publickey".
debug2: fd 6 setting O_NONBLOCK
debug2: fd 7 setting O_NONBLOCK
debug1: channel 0: new session [client-session] (inactive timeout: 0)
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: filesystem
debug3: client_repledge: enter
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: client_input_hostkeys: received RSA key SHA256:Uk7aogRsvRphJsBjM6Q1IQINpOmMyXytdgs9QGqdrCA
debug3: client_input_hostkeys: received ECDSA key SHA256:FeJiekFTTnXWc1GDUfFVcRckyZkZfdmks1dSqtS6OVg
debug3: client_input_hostkeys: received ED25519 key SHA256:dPC6RYiFfvVB/epk5s/lisF4jCbOFJnitbYPy6Dsog8
debug1: client_input_hostkeys: searching /Users/steve.wirt/.ssh/known_hosts for git.drupal.org / (none)
debug3: hostkeys_foreach: reading file "/Users/steve.wirt/.ssh/known_hosts"
debug3: hostkeys_find: found ecdsa-sha2-nistp256 key at /Users/steve.wirt/.ssh/known_hosts:50
debug3: hostkeys_find: found ecdsa-sha2-nistp256 key under different name/addr at /Users/steve.wirt/.ssh/known_hosts:51
debug1: client_input_hostkeys: searching /Users/steve.wirt/.ssh/known_hosts2 for git.drupal.org / (none)
debug1: client_input_hostkeys: hostkeys file /Users/steve.wirt/.ssh/known_hosts2 does not exist
debug3: client_input_hostkeys: 3 server keys: 2 new, 18446744073709551615 retained, 2 incomplete match. 0 to remove
debug1: client_input_hostkeys: host key found matching a different name/address, skipping UserKnownHostsFile update
debug3: client_repledge: enter
debug3: receive packet: type 4
debug1: Remote: /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k:1: key options: command user-rc
debug3: receive packet: type 4
debug1: Remote: /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k:1: key options: command user-rc
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 5 setting TCP_NODELAY
debug3: set_sock_tos: set socket 5 IP_TOS 0x20
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env NVM_RC_VERSION
debug3: Ignored env TERM_PROGRAM
debug3: Ignored env EMAIL_PREFIX
debug3: Ignored env NVM_CD_FLAGS
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug3: Ignored env TMPDIR
debug3: Ignored env HOMEBREW_REPOSITORY
debug3: Ignored env HOMEBREW_SYSTEM_ENV_TAKES_PRIORITY
debug3: Ignored env TERM_PROGRAM_VERSION
debug3: Ignored env PROFILE_UUID
debug3: Ignored env UDID
debug3: Ignored env TERM_SESSION_ID
debug3: Ignored env USER
debug3: Ignored env NVM_DIR
debug3: Ignored env COMMAND_MODE
debug3: Ignored env CIVICHOME
debug3: Ignored env SERIAL_NUMBER
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env DEVICE_ID
debug3: Ignored env __CF_USER_TEXT_ENCODING
debug3: Ignored env MODEL_NAME
debug3: Ignored env TERM_FEATURES
debug3: Ignored env AWS_PROFILE
debug3: Ignored env XDEBUG_CONFIG
debug3: Ignored env TERMINFO_DIRS
debug3: Ignored env PATH
debug3: Ignored env BLUEPRINT_NAME
debug3: Ignored env _
debug3: Ignored env LaunchInstanceID
debug1: channel 0: setting env GIT_PROTOCOL = "version=2"
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env FULL_NAME
debug3: Ignored env __CFBundleIdentifier
debug3: Ignored env AWS_DEFAULT_REGION
debug3: Ignored env PWD
debug3: Ignored env DEPARTMENT
debug3: Ignored env EDITOR
debug3: Ignored env AWS_SECRET_ACCESS_KEY
debug1: channel 0: setting env LANG = "en_US.UTF-8"
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env DEVICE_NAME
debug3: Ignored env ITERM_PROFILE
debug3: Ignored env JOB_TITLE
debug3: Ignored env XPC_FLAGS
debug3: Ignored env MODEL_IDENTIFIER
debug3: Ignored env HISTIGNORE
debug3: Ignored env XPC_SERVICE_NAME
debug3: Ignored env HISTCONTROL
debug3: Ignored env SSH_KEY_PATH
debug3: Ignored env AWS_ACCESS_KEY_ID
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env COLORFGBG
debug1: channel 0: setting env LC_TERMINAL_VERSION = "3.5.14"
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env HOMEBREW_PREFIX
debug3: Ignored env GIT_SSH_COMMAND
debug3: Ignored env ITERM_SESSION_ID
debug3: Ignored env LOGNAME
debug3: Ignored env GOPATH
debug3: Ignored env EMAIL
debug3: Ignored env INFOPATH
debug3: Ignored env HOMEBREW_CELLAR
debug3: Ignored env ASSET_TAG
debug1: channel 0: setting env LC_TERMINAL = "iTerm2"
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env BLUEPRINT_ID
debug3: Ignored env SECURITYSESSIONID
debug3: Ignored env GIT_EXEC_PATH
debug3: Ignored env COLORTERM
debug3: Ignored env CIVICLOGS
debug1: Sending command: git-upload-pack 'project/dkan_dataset_archiver.git'
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug3: client_repledge: enter
debug1: pledge: fork
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
remote: Enumerating objects: 90, done.
remote: Counting objects: 100% (87/87), done.
remote: Compressing objects: 100% (55/55), done.
debug3: receive packet: type 96
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug2: channel 0: rcvd eow
debug2: chan_shutdown_read: channel 0: (i0 o1 sock -1 wfd 6 efd 8 [write])
debug2: channel 0: input open -> closed
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug3: channel 0: will not send data after close
debug2: channel 0: obuf empty
debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 7 efd 8 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send_close2
debug2: channel 0: send close for remote id 0
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
remote: Total 90 (delta 26), reused 81 (delta 21), pack-reused 3 (from 1)
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 [session] r0 nm0 i3/0 o3/0 e[write]/0 fd -1/-1/8 sock -1 cc -1 nc0 io 0x00/0x00)

debug3: send packet: type 1
Transferred: sent 3408, received 32668 bytes, in 0.9 seconds
Bytes per second: sent 3823.6, received 36651.4
debug1: Exit status 0
Receiving objects: 100% (90/90), 21.47 KiB | 3.58 MiB/s, done.
Resolving deltas: 100% (26/26), done.
[steve.wirt] ~/workspace
08:44:21 »

This seems to be the list of public Zscaler IPs

I am not certain if these Firewall config requirements apply.

https://unix.stackexchange.com/questions/758893/ssh-connection-stop-at-d... has a variety of potential causes for hanging after SSH2_MSG_KEXINIT sent

Since this is happening as the SSH key exchange starts, the SSH auth log is likely to be the most useful on our end. I can search those by public key fingerprint, which is in your logs. I only see successes from a single IP:

Aug 28 16:26:44 gitlab1-aws sshd[2556359]: Accepted publickey for git from 47.xxx.xxx.xxx port 49461 ssh2: RSA SHA256:3xjfQwRbu3AxFq1N63QIkLmrYalgwWBb6iBw7np7Fss

That IP must be when you are not using zscalar. That’s consistent with your logs when using zscalar not getting to the phase where the public key is offered. That’s the first log entry for any given connection, so your connection may not be getting as far as something that might be logged on our end. https://config.zscaler.com/zscaler.net/hubs is too many IP ranges for me to be able to practically search for.

We aren’t doing anything to specifically block zscalar. Does zscaler support offer any help?

Thanks @drumm I appreciate the help and will take your findings back to Zscaler and see if they can go farther.

I will report back here when I get a response.

This has been resolved on our agency's instance of Zscaler. Apparently port 22 was on blocked on Zscaler for the destination.

Changing this to Closed (outdated).