Basic Handshake protocol

I wrote a basic handshake protocol in Rust. Let's break down the entire implementation step-by-step, explaining each component and how it addresses the requirements for a secure handshake protocol.

A. Protocol Design Overview

The protocol implements mutual authentication using pre-shared keys (PSK) with the following message sequence:

• ClientHello:
  
  Client ID (variable length)
  
  Client Nonce (16-byte random value)




• ServerHello:
  
  Server Nonce (16-byte random value)
  
  HMAC(PSK, ClientNonce + ServerNonce + "Server")




• ClientAuth:
  
  HMAC(PSK, ClientNonce + ServerNonce + "Client")




• ServerAck:
  
  "AUTH_SUCCESS" confirmation

B. Security Features Implemented:

• Mutual Authentication: Both client and server prove knowledge of PSK


• Replay Attack Prevention: Unique nonces for each session


• Message Integrity: HMAC-SHA256 protects against tampering


• Context Separation: Different HMAC contexts for client/server roles


• Freshness Guarantee: Nonces ensure responses are current

C. Key Components Explained:

a. KeyStore (protocol.rs)

#[derive(Clone)]

pub struct KeyStore(HashMap<String, Vec<u8>>);

• Purpose: Stores pre-shared keys for clients


• Implementation:
  
  
  
  
  • Uses HashMap for client ID → key mapping
  
  
  • 
    #[derive(Clone)] enables safe sharing between threads
  
  
  • 
    get_key() retrieves key with error handling
  
  
  
  
  
  

b. Nonce Generation

pub fn generate_nonce() -> [u8; NONCE_SIZE] {

    let mut nonce = [0u8; NONCE_SIZE];

    rand::rngs::OsRng.fill_bytes(&mut nonce);

    nonce

}

• Security Importance:
  
  
  
  
  • Prevents replay attacks
  
  
  • Ensures session uniqueness
  
  
  
  
  
  



• Implementation:
  
  
  
  
  • Uses cryptographically secure OsRng
  
  
  • Generates 16 random bytes (128 bits)
  
  
  
  
  
  

c. HMAC Operations

pub fn compute_hmac(key: &[u8], data: &[&[u8]]) -> Result<[u8; 32]> {

    let mut mac = HmacSha256::new_from_slice(key)?;

    for d in data {

        mac.update(d);

    }

    Ok(mac.finalize().into_bytes().into())

}

• Security Properties:
  
  
  
  
  • Keyed-Hash Message Authentication Code
  
  
  • SHA-256 provides collision resistance
  
  
  
  
  
  



• Implementation Notes:
  
  
  
  
  • Concatenates inputs: [c_nonce, s_nonce, role]
  
  
  • Role strings ("Client"/"Server") prevent reflection attacks
  
  
  
  
  
  

D. Server Implementation (server.rs)

a. Initialization

let listener = TcpListener::bind("127.0.0.1:8080").await?;

let keystore = Arc::new(KeyStore::new());

• Concurrency Handling:
  
  
  
  
  • Arc enables thread-safe reference counting
  
  
  • Allows sharing keystore across connections
  
  
  
  
  
  

b. Connection Handling

tokio::spawn(async move {

    // Handshake logic

});

• Asynchronous Design:
  
  
  
  
  • Uses Tokio runtime for async I/O
  
  
  • Spawns new task per connection
  
  
  
  
  
  

c. Handshake Sequence

i. Receive ClientHello:

let client_id_len = socket.read_u8().await?;

socket.read_exact(&mut client_id_buf[..client_id_len]).await?;

socket.read_exact(&mut c_nonce).await?;

• Reads client ID (length-prefixed)


• Reads 16-byte client nonce

ii. Send ServerHello:

let server_hmac = compute_hmac(psk, &[&c_nonce, &s_nonce, b"Server"])?;

socket.write_all(&s_nonce).await?;

socket.write_all(&server_hmac).await?;

• Proves server knows PSK


• Binds response to client's nonce

iii. Verify ClientAuth:

verify_hmac(psk, &[&c_nonce, &s_nonce, b"Client"], &client_hmac)?;

• Validates client knows PSK


• Ensures client received server's nonce

iv. Send Acknowledgement:

socket.write_all(b"AUTH_SUCCESS").await?;

• 12-byte fixed message confirms auth

E. Client Implementation (client.rs)

a. Handshake Sequence

i. Send ClientHello:

socket.write_u8(client_id.len() as u8).await?;

socket.write_all(client_id.as_bytes()).await?;

socket.write_all(&c_nonce).await?;

• Length-prefixed client ID


• Random client nonce

ii. Verify ServerHello:

verify_hmac(psk, &[&c_nonce, &s_nonce, b"Server"], &server_hmac)?;

• Validates server identity


• Ensures response is fresh (contains client nonce)

iii. Send ClientAuth:

let client_hmac = compute_hmac(psk, &[&c_nonce, &s_nonce, b"Client"])?;

socket.write_all(&client_hmac).await?;

• Proves client knows PSK


• Binds response to server's nonce

iv. Verify ServerAck:

if &ack == b"AUTH_SUCCESS" {

    println!("Authenticated with server!");

}

• Final confirmation of successful auth

F. Protocol Security Analysis

a. Replay Attack Prevention

• Client Nonce: Ensures server response is fresh


• Server Nonce: Ensures client authentication is fresh


• HMAC Context: Includes both nonces in all authentication tags

b. Authentication Strength

• Mutual Authentication: Both parties prove PSK knowledge


• Context Separation: Different HMAC strings for client/server roles


• Key Binding: All HMACs incorporate both parties' nonces

c. Message Sequencing

• Strict state machine enforced by:
  
  
  
  
  • Server waits for ClientHello before responding
  
  
  • Client waits for ServerHello before authenticating
  
  
  • Server waits for ClientAuth before acking
  
  
  
  
  
  

d. Minimal Security Concepts

• Authentication: HMAC proves PSK knowledge


• Integrity: HMAC protects against message tampering


• Freshness: Nonces guarantee message recency


• Key Confidentiality: PSK never transmitted

This implementation provides a solid foundation for secure protocol design, demonstrating core concepts like mutual authentication, nonce-based freshness, and HMAC-based message integrity. The Rust implementation leverages async I/O for performance and type safety for correctness.