Detect and mask sensitive information in your application
Netra SDK provides advanced PII detection with multiple engines:To use the PII detection features provided by Netra SDK:
pip install 'netra-sdk[presidio]'
poetry add netra-sdk --extras "presidio"
If you try to use the PII detection utility in Netra SDK without installing this dependency, you will receive an exception. So, always make sure to install the package if you plan to use this utility.
Default PII Detector (Recommended)
from netra.pii import get_default_detector# Get default detector with custom settingsdetector = get_default_detector( action_type="MASK", # Options: "BLOCK", "FLAG", "MASK" entities=["EMAIL_ADDRESS", "PERSON", "PHONE_NUMBER"])# Detect PII in texttext = "Contact John Doe at john.doe@example.com or call him at 555-123-4567"result = detector.detect(text)print(f"Has PII: {result.has_pii}")print(f"Masked text: {result.masked_text}")print(f"Entity mappings: {result.entities}")
Entity Anonymization Details
Netra SDK uses different anonymization mechanisms based on entity type:
Important Note: The hashing mechanism is deterministic, meaning that the same entity will always produce the same hash value. This ensures consistency across multiple detections and allows for proper data correlation while maintaining privacy.
Format: <ENTITY_TYPE_hash>
Examples:
Person: John Doe → <PERSON_6cea57c2>
Phone: 555-123-4567 → <PHONE_NUMBER_2dcbf551>
Example Results
Original text:
Contact John Doe at john.doe@example.com or call him at 555-123-4567
Anonymized text:
Contact <PERSON_6cea57c2> at fank.fzr@hgbcfxa.zrl or call him at 55<PHONE_NUMBER_2dcbf551>
Entity mappings:
PHONE_NUMBER_2dcbf551 -> 5-123-4567 fank.fzr@hgbcfxa.zrl -> john.doe@example.com PERSON_6cea57c2 -> John Doe
Verification Features
Emails maintain format (contains @ and .)
Hashes follow consistent naming convention
Entity types are preserved in anonymization
Consistent mapping between original and anonymized values
Presidio-based Detection
from netra.pii import PresidioPIIDetector# Initialize detector with different action typesdetector = PresidioPIIDetector( action_type="MASK", # Options: "FLAG", "MASK", "BLOCK" score_threshold=0.8, entities=["EMAIL_ADDRESS", "PERSON", "PHONE_NUMBER"])# Detect PII in texttext = "Contact John Doe at john.doe@example.com"result = detector.detect(text)print(f"Has PII: {result.has_pii}")print(f"Masked text: {result.masked_text}")print(f"Entity mappings: {result.entities}")
from netra.pii import get_default_detector# Get default detector with custom settingsdetector = get_default_detector( action_type="MASK" # Options: "BLOCK", "FLAG", "MASK")# Works with chat message formatschat_messages = [ {"role": "user", "content": "My email is john@example.com"}, {"role": "assistant", "content": "I'll help you with that."}, {"role": "user", "content": "My phone is 555-123-4567"}]result = detector.detect(chat_messages)print(f"Masked messages: {result.masked_text}")
Custom Models for PII Detection
The PresidioPIIDetector supports custom NLP models through the nlp_configuration parameter, allowing you to use specialized models for improved PII detection accuracy. You can configure custom spaCy, Stanza, or transformers models:
NLP Configuration Example
Follow this configuration structure to provide your custom models.
nlp_configuration = { "nlp_engine_name": "spacy|stanza|transformers", "models": [ { "lang_code": "en", # Language code "model_name": "model_identifier" # Varies by engine type } ], "ner_model_configuration": { # Optional, mainly for transformers # Additional configuration options }}
Using Custom spaCy Models
from netra.pii import PresidioPIIDetector# Configure custom spaCy modelspacy_config = { "nlp_engine_name": "spacy", "models": [{"lang_code": "en", "model_name": "en_core_web_lg"}]}detector = PresidioPIIDetector( nlp_configuration=spacy_config, action_type="MASK", score_threshold=0.8)text = "Dr. Sarah Wilson works at 123 Main St, New York"result = detector.detect(text)print(f"Detected entities: {result.pii_entities}")
Using Stanza Models
from netra.pii import PresidioPIIDetector# Configure Stanza modelstanza_config = { "nlp_engine_name": "stanza", "models": [{"lang_code": "en", "model_name": "en"}]}detector = PresidioPIIDetector( nlp_configuration=stanza_config, action_type="FLAG")text = "Contact Alice Smith at alice@company.com"result = detector.detect(text)print(f"PII detected: {result.has_pii}")
Using Transformers Models
For advanced NER capabilities, you can use transformer-based models: