Skip to content

a-abdellatif98/ff1_gem

BranchesTags

Repository files navigation

FF1 Format Preserving Encryption Gem

A Ruby implementation of the FF1 Format Preserving Encryption algorithm from NIST SP 800-38G.

Overview

Format Preserving Encryption (FPE) allows you to encrypt data while maintaining its original format. This is particularly useful when you need to encrypt sensitive data like credit card numbers, social security numbers, or other structured data while keeping the same format for compatibility with existing systems.

The FF1 algorithm is one of two methods specified in NIST Special Publication 800-38G for Format-Preserving Encryption.

Features

  • Full implementation of NIST FF1 algorithm
  • Dual-mode operation: Reversible and Irreversible encryption
  • Support for any radix from 2 to 65, 536
  • Full UTF-8 text encryption support - encrypt arbitrary text while maintaining FF1 security properties
  • Rails/ActiveRecord integration - seamless database encryption with GDPR compliance
  • Simplified soft delete - Uses single deleted_at timestamp column for tracking
  • Tweak support for additional security
  • GDPR "right to be forgotten" compliance with soft delete
  • Automatic encryption/decryption in Rails models
  • Rails migration generators and configuration
  • Query scopes for active/deleted records
  • Restore functionality for soft-deleted records
  • Proper input validation and error handling
  • Comprehensive test suite
  • Thread-safe implementation

Installation

Add this line to your application's Gemfile:

gem 'ff1'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install ff1

Usage

Basic Usage

require 'ff1'

# Create a cipher with a 128-bit key for decimal numbers (radix 10)
key = "\x2B\x7E\x15\x16\x28\xAE\xD2\xA6\xAB\xF7\x15\x88\x09\xCF\x4F\x3C"

# Reversible mode (default) - can decrypt back to original
cipher = FF1::Cipher.new(key, 10, FF1::Modes::REVERSIBLE)
plaintext = "4111111111111111"
ciphertext = cipher.encrypt(plaintext)
decrypted = cipher.decrypt(ciphertext)
puts "#{plaintext} → #{ciphertext} → #{decrypted}"
# => "4111111111111111 → 8224999410799188 → 4111111111111111"

# Irreversible mode - cannot decrypt (for GDPR compliance)
secure_cipher = FF1::Cipher.new(key, 10, FF1::Modes::IRREVERSIBLE)  
secure_ciphertext = secure_cipher.encrypt(plaintext)
# secure_cipher.decrypt(secure_ciphertext)  # ❌ Raises error
puts "Securely deleted: #{plaintext} → #{secure_ciphertext}"
# => "Securely deleted: 4111111111111111 → 2858907702179518"

Using Tweaks

Tweaks provide additional input to the encryption algorithm for enhanced security:

cipher = FF1::Cipher.new(key, 10)
tweak = "user123"

plaintext = "1234567890"
ciphertext = cipher.encrypt(plaintext, tweak)
decrypted = cipher.decrypt(ciphertext, tweak)

Different Radix Values

# For hexadecimal data (radix 16)
hex_cipher = FF1::Cipher.new(key, 16)
hex_data = "ABCDEF123456"
encrypted_hex = hex_cipher.encrypt(hex_data)

# For binary data (radix 2)
binary_cipher = FF1::Cipher.new(key, 2)
binary_data = "1010101"  # Must be long enough to meet domain requirements
encrypted_binary = binary_cipher.encrypt(binary_data)

Text Encryption (NEW!)

The gem now supports encrypting arbitrary UTF-8 text while maintaining security properties:

cipher = FF1::Cipher.new(key, 10)

# Encrypt any text including Unicode and special characters
text = "Hello, World! 🌍 Special chars: @#$%^&*()"
encrypted_text = cipher.encrypt_text(text)
decrypted_text = cipher.decrypt_text(encrypted_text)

puts encrypted_text  # Returns base64-encoded encrypted data
# => "SGVsbG8sIFdvcmxkISDwn42NIFNwZWNpYWwgY2hhcnM6IEAjJCVeJiooKQ=="

# Text with tweak for additional security
context = "user_session_123"
encrypted_with_context = cipher.encrypt_text(text, context)
decrypted_with_context = cipher.decrypt_text(encrypted_with_context, context)

# Irreversible text encryption for GDPR compliance
irreversible_cipher = FF1::Cipher.new(key, 10, FF1::Modes::IRREVERSIBLE)
sensitive_data = "Personal information to be securely deleted"
irreversibly_encrypted = irreversible_cipher.encrypt_text(sensitive_data)
# Cannot decrypt - data is permanently transformed while maintaining consistency

ActiveRecord Integration (NEW!)

The gem now includes full Rails/ActiveRecord integration for seamless database encryption with GDPR compliance features.

Installation for Rails

Add the gem to your Gemfile:

gem 'ff1'

Run the generator to create the configuration file:

rails generate ff1:install

This creates config/initializers/ff1.rb with configuration options.

Configuration

Configure FF1 in your Rails initializer:

# config/initializers/ff1.rb
FF1::ActiveRecord.configure do |config|
  # Required: Set your encryption key (store securely!)
  config.global_key = Rails.application.credentials.ff1_encryption_key
  # Or use environment variable:
  # config.global_key = ENV['FF1_ENCRYPTION_KEY']&.b
  
  # Optional: Set default encryption mode
  config.default_mode = FF1::Modes::REVERSIBLE
  
  # Optional: Customize column name for soft delete tracking
  config.deleted_at_column = :deleted_at
end

Basic Model Integration

Include FF1::ActiveRecord in your models and configure encrypted columns:

class User < ApplicationRecord
  include FF1::ActiveRecord
  
  # Encrypt email and phone with reversible encryption (can decrypt)
  ff1_encrypt :email, :phone, mode: :reversible
  
  # Encrypt sensitive data with irreversible encryption (cannot decrypt - GDPR compliant)
  ff1_encrypt :ssn, :address, :full_name, mode: :irreversible
  
  # Optional: Define scope for active users (alternative to built-in ff1_active)
  scope :active, -> { where(deleted_at: nil) }
end

Database Migration

Generate and run migration to add soft delete columns:

rails generate ff1:install User
rails db:migrate

This adds a deleted_at timestamp column to your users table for tracking soft deletes.

Usage Examples

# Create user - data is automatically encrypted before saving
user = User.create!(
  name: 'John Doe',
  email: '[email protected]',
  phone: '555-1234', 
  ssn: '123-45-6789'
)

# Access data - reversible columns decrypt automatically  
puts user.email  # => '[email protected]' (decrypted)
puts user.phone  # => '555-1234' (decrypted)
puts user.ssn    # => '[ENCRYPTED]' (irreversible, cannot decrypt)

# Data is encrypted in the database
User.connection.select_value("SELECT email FROM users WHERE id = #{user.id}")
# => "8224999410799188" (encrypted format)

GDPR Compliant Soft Delete

The integration provides GDPR-compliant "right to be forgotten" functionality:

# Soft delete with irreversible encryption
user.destroy  # Encrypts all sensitive data irreversibly, keeps record

# Check deletion status
user.ff1_deleted?     # => true (checks if deleted_at is set)
user.ff1_deleted_at   # => 2023-01-01 12:00:00 UTC

# Data is now irreversibly encrypted
user.email  # => '[ENCRYPTED]'
user.phone  # => '[ENCRYPTED]' 
user.ssn    # => '[ENCRYPTED]'

# Restore record (but data remains encrypted)
user.ff1_restore
user.ff1_deleted?  # => false (deleted_at is cleared)

# Find deleted users and restore them
deleted_user = User.ff1_deleted.first
deleted_user.ff1_restore

# True hard delete if needed
user.ff1_hard_destroy!

Query Scopes

Use built-in scopes to query active vs soft-deleted records:

# Active (non-deleted) users only
User.ff1_active.count         # => 150
User.ff1_active.where(...)    # Chain with other scopes

# Soft-deleted users only  
User.ff1_deleted.count        # => 25

# All users (active + deleted)
User.ff1_all.count            # => 175

# Recently deleted users
User.ff1_recently_deleted(within: 30.days)

# Old deleted users (for purging)
User.ff1_old_deleted(older_than: 1.year)

Bulk Operations

# Bulk soft delete with encryption
User.ff1_destroy_all(status: 'inactive')  # Returns count of deleted records

# Purge old soft-deleted records
User.ff1_purge_deleted(older_than: 2.years)  # Permanent deletion

# Batch encrypt existing data
User.ff1_encrypt_existing_data!(columns: [:email, :phone])

# Statistics
User.ff1_stats
# => {
#   total_records: 175,
#   active_records: 150, 
#   deleted_records: 25,
#   encrypted_columns: [:email, :phone, :ssn],
#   deletion_rate: 14.29
# }

Advanced Configuration

class User < ApplicationRecord
  include FF1::ActiveRecord
  
  # Per-column configuration
  ff1_encrypt :email, 
    mode: :reversible,
    key: Rails.application.credentials.user_email_key,  # Custom key
    radix: 256  # For text data
  
  ff1_encrypt :credit_card_number,
    mode: :reversible, 
    radix: 10   # For numeric data
    
  ff1_encrypt :sensitive_notes,
    mode: :irreversible,  # Cannot be decrypted
    radix: 256
end

Thread Safety

The ActiveRecord integration is thread-safe and caches cipher instances for performance:

# Safe for concurrent access
User.transaction do
  User.create!(email: '[email protected]')
  User.create!(email: '[email protected]')  
end

Performance Considerations

  1. Cipher Caching: Ciphers are cached per column configuration for performance
  2. Bulk Operations: Use ff1_find_each for processing large datasets
  3. Indexing: Encrypted data cannot be indexed for searching - plan accordingly
  4. Query Performance: Use the provided scopes which include proper indexes

GDPR Compliance Features

The ActiveRecord integration specifically supports GDPR requirements:

  1. Right to be Forgotten: destroy method irreversibly encrypts sensitive data
  2. Data Minimization: Only configured columns are encrypted
  3. Audit Trail: Soft delete maintains record for compliance tracking
  4. Data Portability: Reversible encryption allows data export when legally required

Troubleshooting

Migration Issues

If you encounter errors during migration generation, ensure you're running the correct commands:

# Generate configuration (run once)
rails generate ff1:install

# Generate migration for specific models
rails generate ff1:install User Post Comment
rails db:migrate

Stack Overflow on Destroy

If you encounter "stack level too deep" errors when calling destroy, ensure your migration has been run and the deleted_at column exists:

# Check if column exists
User.column_names.include?('deleted_at')  # Should return true

# If missing, run the migration
rails generate ff1:install User
rails db:migrate

Restoring Deleted Records

Remember that irreversibly encrypted data cannot be recovered:

# This will restore the record but encrypted data stays encrypted
deleted_user = User.ff1_deleted.first
deleted_user.ff1_restore

# Reversible data might be recoverable if it was encrypted before deletion
puts deleted_user.email  # May show original data or '[ENCRYPTED]'

Requirements

Ruby Version:

  • Ruby 3.2.0 or higher

Dependencies:

  • ActiveRecord ~> 7.0 (for Rails integration)

Key Requirements

The FF1 algorithm supports AES key lengths:

  • 128-bit keys (16 bytes)
  • 192-bit keys (24 bytes)
  • 256-bit keys (32 bytes)
# 128-bit key
key_128 = SecureRandom.bytes(16)
cipher_128 = FF1::Cipher.new(key_128, 10)

# 256-bit key
key_256 = SecureRandom.bytes(32)
cipher_256 = FF1::Cipher.new(key_256, 10)

Domain Size Requirements

For security, the FF1 algorithm requires that radix^length >= 100 . For better security, radix^length >= 1,000,000 is recommended.

Examples:

  • Decimal (radix 10): minimum 2 digits, recommended 7+ digits
  • Hexadecimal (radix 16): minimum 2 digits, recommended 5+ digits
  • Binary (radix 2): minimum 7 bits, recommended 20+ bits

Error Handling

The gem provides comprehensive error handling:

begin
  cipher = FF1::Cipher.new(key, 10)
  result = cipher.encrypt("123")  # Too short
rescue FF1::Error => e
  puts "Encryption error: #{e.message}"
end

Common errors:

  • FF1::Error: Base error class for all FF1-related errors
  • Invalid key length
  • Invalid radix (must be 2-65536)
  • Input too short or empty
  • Invalid characters for the specified radix
  • Domain size too small

Security Considerations

  1. Key Management: Use cryptographically secure random keys and protect them appropriately
  2. Domain Size: Ensure your domain size meets the minimum requirements (preferably >= 1,000,000)
  3. Tweaks: Use tweaks when possible for additional security
  4. Input Validation: The gem validates inputs, but ensure your data meets the requirements

Algorithm Details

This implementation follows NIST SP 800-38G specification for the FF1 algorithm:

  • Uses 10 rounds of a Feistel network
  • Supports any radix from 2 to 65, 536
  • Uses AES as the underlying block cipher
  • Implements proper padding and round function as specified

Thread Safety

The FF1:: Cipher instances are thread-safe for encryption and decryption operations. However, you should not modify the cipher instance (key, radix) from multiple threads simultaneously.

Changelog

Version 1.2.6+

  • BREAKING CHANGE: Simplified soft delete to use only deleted_at column
  • Removed ff1_deleted boolean column requirement
  • Fixed "stack level too deep" error in destroy method
  • Updated all scopes to work with deleted_at timestamp only
  • Improved migration generator to create correct class names
  • Updated ff1_deleted? method to check deleted_at.present?
  • Enhanced restore functionality and documentation

Previous Versions

  • Added Rails/ActiveRecord integration
  • Added GDPR compliance features
  • Added text encryption support
  • Initial FF1 algorithm implementation

Contributing

Bug reports and pull requests are welcome on GitHub.

License

The gem is available as open source under the MIT License.

References

About

A Ruby implementation of the FF1 Format Preserving Encryption algorithm from NIST SP 800-38G.

Resources

Readme

License

MIT license
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

2 tags

Packages

No packages published

Languages