4

I am trying to create a DTLS "connection" using Schannel under Windows (I am testing under recent Windows 10 version, so all DTLS versions supported by Schannel should be available)

I tried starting from working code to establish a regular TLS connection by following the documentation:

  1. InitializeSecurityContext with null input on the first pass, SECBUFFER_TOKEN & SECBUFFER_ALERT on output
  2. AcceptSecurityContext with SECBUFFER_TOKEN & SECBUFFER_EMPTY on input, SECBUFFER_TOKEN & SECBUFFER_ALERT on output.
  3. Repeat the two steps until they succeed, and then move on to using Encrypt/DecryptMessage

This works perfectly fine in stream mode (ISC_REQ_SEQUENCE_DETECT | ISC_REQ_REPLAY_DETECT | ISC_REQ_CONFIDENTIALITY | ISC_RET_EXTENDED_ERROR | ISC_REQ_ALLOCATE_MEMORY | ISC_REQ_STREAM)

If I try to substitute STREAM with ISC/ASC_REQ_DATAGRAM, my InitializeSecurityContext succeeds with SEC_I_CONTINUE_NEEDED as expected, but my very first AcceptSecurityContext then fails with SEC_E_INVALID_PARAMETER.

I have tried setting grbitEnabledProtocols of my SCHANNEL_CRED to 0 to use the defaults as documented on both sides, I also tried setting it to SP_PROT_DTLS1_X, and I still get the Invalid Parameter return from my first ASC. I have also tried the DTLS_1_0 constants just in case.

Also, all security protocols are enabled by default in my registry settings.

From my understanding of the DTLS RFC, my code is failing at the HelloVerifyRequest step, and, again from my understanding of the RFC, this part requires that the security provider generates a cookie from a few parts of the ClientHello message as well as the source's IP address. However, I could not find any documented way to pass this information to the ASC function.

(I think? :) ) I have searched the entire internet for any working example usage of DTLS under Schannel without any luck. On stackoverflow, I found this question that simply mentions that it is supported: Is DTLS supported by Schannel on Windows 7?, and the linked MSDN article is just a high level overview.

I searched for any usage of the constants that are related to this feature... I searched for any usage of the constants that are related to this (ISC_REQ_DATAGRAM, SP_PROT_DTLS*, SECBUFFER_DTLS_MTU, ...) and the only thing I could find on all search engines I could think of were either copies of sspi.h or sites that index the constants in the Windows API...

I know DTLS is well supported by other implementations (OpenSSL etc), but I would really prefer to stay with Schannel, as other parts of my code currently work just fine with Schannel in TLS mode.

fbrosseau
  • 325
  • 2
  • 7

2 Answers2

3

From Microsoft: There is no documentation for implementing DTLS using Schannel. Known and persistent doc gap.

There are a few differences, but a TLS client or server can be adapted to DTLS fairly easily (a number of customers have done this successfully).

  1. Set SCHANNEL_CRED.grbitEnabledProtocols to SP_PROT_DTLS1_X.
  2. When calling AcceptSecurityContext, pass the client’s SOCKADDR via SECBUFFER_EXTRA.
  3. MTU can be set via SetContextAttributes using constant SECPKG_ATTR_DTLS_MTU where the buffer is just an pointer to a ULONG. [Default is 1096 bytes.]
  4. When ISC/ASC return SEC_I_MESSAGE_FRAGMENT, send this fragment and call ISC/ASC again, in a loop, to get the next fragment (without trying to read data from the network).
  5. Implement timeout and retransmit logic in your application (since schannel does not own the socket).
  6. When receiving fragments, schannel will attempt to eliminate duplicates, re-order and re-assemble, if possible.
  7. SCHANNEL_SHUTDOWN does not apply to DTLS.
Matthew J. Bobowski
  • 46
  • 6
  • Considering opening a new question, but maybe you can answer here? After I receive SEC_I_MESSAGE_FRAGMENT from ASC, and call it again to retrieve the next fragment, I am getting SEC_E_INCOMPLETE_MESSAGE and nothing in the output token buffer. (This is when passing a zero-length input token buffer - if I re-pass the original input, I get a repeat of the first fragment as output.) Is there something specific I need to do to indicate to ASC that it should give me the next fragment? – Haddon CD. May 07 '21 at 17:29
  • I take it you are still in your negotiation loop? If so, exit the loop and process the error for encrypted data underflow, queue and process on next recv. To do this check for SECBUFFER_EXTRA from index 1 (second element) returned from ASC. Backup data to remove already processed information from the input buffer. When receiving data again add new data to the backed up data by resizing the stored data buffer and either continue negotiation or call DecryptMessage(). – Matthew J. Bobowski May 21 '21 at 07:27
-1

You can use https://github.com/mobius-software-ltd/iotbroker.cloud-windows-client As a sample to implement DTLS on windows It does not uses SChannel but netty library. MQTT-SN And CoAP are both supporting DTLS under this project. BR Yulian Oifa

Yulian Oifa
  • 83
  • 7
  • This question was specifically about SChannel. Outside of SChannel, DTLS can trivially be implemented using OpenSSL and others. Most of your stackoverflow answers appear to be advertisements for your projects. – fbrosseau Feb 06 '21 at 19:23