Card Troubleshooting
Version 1.0 | July 2025

πŸ’³ DEBIT CARD TROUBLESHOOTING GUIDE

Comprehensive monitoring and troubleshooting for Debit Card file processing

πŸ“‹ Quick Start: Monitor file uploads at 4PM, verify Slack alerts at 8PM and 8:45PM, and confirm morning reports. Follow escalation procedures for any failures or missing files.

πŸš€ Quick Navigation

πŸ“‹ Process Overview

Purpose: This guide provides systematic monitoring and troubleshooting steps for Debit Card file processing. The process involves multiple checkpoints throughout the day to ensure proper file upload, processing, and confirmation.

Key Monitoring Times: 4:00 PM (file upload), 8:00 PM (verification), 8:45 PM (processing confirmation), and next morning (final confirmation). Each checkpoint has specific actions and escalation procedures.

πŸ“€ File Upload Monitoring

1
Initial File Upload Process
4:00 PM

Objective: Bank team initiates the process by uploading subject files to the system starting from 4:00 PM.

What happens at this stage:

  • βœ… Bank team uploads debit card order files to the system
  • βœ… Files are processed and stored in designated directories
  • βœ… System begins initial validation of uploaded files
  • βœ… Monitoring systems start tracking file upload status

Note: No action required from support team at this stage. This is informational to understand the process flow.

πŸ’¬ Slack Verification

2
Slack Channel Monitoring for File Status
8:00 PM

Objective: Verify whether files were successfully uploaded by checking the Slack channel #card-order-monitoring at 8:00 PM.

Expected Slack Alert Indicators:

Successful Upload: Green tick mark with status "Received" and file count
slack image sample for green tick mark
Failed Upload: Red cross mark indicating file was "not received"
slack image sample for red cross mark

βœ… Success Scenario: If files show green tick with "Received" status, proceed to monitor Step 3 at 8:45 PM.

❌ Failure Scenario: If files show red cross or "not received" status, immediately proceed to troubleshooting scenarios below.

πŸ” File Upload Troubleshooting Scenarios

No File Received

Check the following before escalating:

πŸ–₯️ Mount Point Verification Commands

🌐 Environment Access Details

UG Environment

Server: cont16059
User: su – extio_user
Password: ********
Escalate: sudo su - mount -a

Usage: Testing and development environment. Safe for experimental commands and configuration changes.

KE Environment

Server: cont17131
Connection: ssh apizone@10.0.3.11 -p7779
Password: ********
sudo su - mount -a

⚠️ Caution: Production environment. Make changes carefully and always backup configurations before modifications.

πŸ“‹ Check Debit Card Upload POD Logs
# Check if any xlsx files were received today
          pods | grep debit
          kubectl logs (pod-name) 
          

Action: If no file is received and no errors found, send email to bank team using the template in the Email Templates section.

Email Server Error

When email server returns errors:

πŸ“§ Check ext-mail POD Logs
# Check ext-mail POD for errors
pods | grep mail
kubectl logs (pod-name)

Action: Copy the exact error message from logs and include in email to bank team about Email Server Error.

COS Mount Point Not Mounted

When mount point fails:

log into server (17131 for KE) and 12957 for UG mount -a

Action: If mount command returns errors, copy the exact error message and include in email to bank team about Sybrin COS Mount Point issue.

No Device Production Report

When next day alert shows no Device Production Report:

Action: Send email to PineLabs team and copy bank team for the respective country using the template provided.

⚑ Processing Alerts

3
File Processing Confirmation
8:45 PM

Objective: Once files are processed at 8:45 PM, verify that another Slack alert is triggered with processing confirmation.

Processing Success: Green tick mark with status changed to "Ordered" and count displayed

βœ… Success Scenario: If status shows "Ordered" with count, processing is successful. Proceed to morning confirmation.

slack image sample for green tick mark

❌ Failure Scenario: If no alert or failure status, check upload pod logs for xlsx files and account numbers with date timestamp. Ask bank team to verify and send account number for clearance from MSB.

slack image sample for green tick mark
πŸ“‹ Upload POD Log Analysis

πŸŒ… Morning Confirmation

4
Next Morning Verification
Next Morning

Objective: Verify the confirmation email received next morning with subject line format: "KE MasterCard Debit Card Report File Downloaded: [Timestamp]"

Expected Email Format:

πŸ“§ Morning Confirmation Email
Subject: KE MasterCard Debit Card Report File Downloaded: Fri Jul 11 08:00:07 EAT 2025

Content: [Country name] [Timestamps] [File processing details]

Cross-verification Required: Compare the morning email details with the previous day's Slack alerts to ensure consistency.

Zero Count Troubleshooting:

If count is 0 for any country:

  1. Review upload pod logs for failure causes
  2. Check card pack ID in database table debit_card_monitoring
  3. Ensure ordered_timestamp is set for yesterday's date
πŸ—„οΈ Database Verification for Kenya
# For Kenya - check from 8:00 AM current date
# Extract card pack ID from logs
kubectl logs upload-pod-name | grep -i "card.pack" | grep "$(date '+%Y-%m-%d')"

# Check in database table
SELECT card_pack_id, ordered_timestamp, status 
FROM debit_card_monitoring 
WHERE DATE(ordered_timestamp) = CURDATE() - INTERVAL 1 DAY
AND country = 'KE';

Missing records: If records is missing for other dates, send email to PineLabs team using the provided template.

After pinelabs placing file: 1. check file in /DOWNLOAD/ISVC03 and check if it has records or is it empty , 2. If Pinelabs team process the file and places the file in sftp server , check for the respective country in path /DOWNLOAD/ISVC03/ , if file is placed and is in any other directory rather than mentioned above , mail pinelabs to place device production report file in above mentioned path

.

πŸ“§ Email Templates

Scenario 1: Sybrin COS File Not Received

Scenario 2: Email Server Error

Scenario 3: Sybrin COS Mount Point Error

Scenario 4: Missing Device Production Report

πŸ“ž Escalation Procedures

πŸ‡°πŸ‡ͺ KENYA

Kenya Contact List

πŸ‡ΉπŸ‡Ώ TANZANIA

Tanzania Contact List

πŸ‡ΊπŸ‡¬ UGANDA

Uganda Contact List

CC Recipients for all emails:

Escalation Timeline: