Micro Hardening Training Participation Report

A report on participating in Micro Hardening security training, defending an EC site through attack observation, tamper detection, and attack surface reduction.

A record of participating in the “Micro Hardening” security training.

Training Overview

Teams of 4 defend an actual EC site (WordPress + Apache + MySQL running on a single server) to maximize sales. A crawler makes purchases on the EC site every minute, and teams compete on total purchases over 45 minutes. Sales decrease from attack-induced tampering, and successful defenses earn bonus points.

The training consisted of 4 sets of 45 minutes each.

Specific vulnerabilities are not described to prevent spoilers. This summary focuses on our team’s thinking and learnings.

Approach for Each Set

Set 1

At first, we didn’t know the full scope of attack methods and vulnerabilities, so we prioritized understanding “what attacks were coming.” If a service went down, we only performed recovery and treated this as an observation phase.

  • Checked access logs (filtering out our own IP)
    sudo tail -n 500 /var/log/httpd/access_log | grep -v "192.168.0.200"
    
  • Checked access users with the last command
  • Checked for tampering in /var/www/html
  • Compressed access logs into a zip file and submitted them to AI (ChatGPT) for analysis and advice

Set 2

From Set 1’s observations, we identified “SSH-based intrusion and tampering” and “WordPress vulnerabilities being exploited to extract configuration files.” We focused on strengthening tamper detection and recovery systems to prevent damage escalation.

  • Backed up /var/www/html and used diff to detect tampering
    cp -r /var/www/html /tmp/html
    sudo diff -r /var/www/html /tmp/html
    
  • Immediately restored upon detecting tampering
  • Also dumped and backed up MySQL data
    mysqldump -u root -p wordpress > /tmp/wordpress_backup.sql
    
  • Continued log monitoring

Set 3

From observations through Set 2, we identified “FTP-based attacks,” “exploitation of unnecessary plugins,” and “PUT method abuse” – there were too many unnecessary entry points available to attackers. We focused on reducing the attack surface by cleaning up and restricting unnecessary services and accounts.

  • Identified and disabled unnecessary WordPress plugins
  • Stopped FTP service since it was being abused
    sudo service vsftpd stop
    
  • Locked unnecessary accounts and changed passwords for active accounts
    sudo passwd -l admin
    
  • Disabled the PUT method (not needed for normal operations) which was being exploited

We removed and stopped services and plugins one at a time, checking the impact each time.

Set 4

We thought we had closed the main backdoors and attack vectors from previous sets, but an anomaly occurred where only specific products could not be purchased. We determined that “data tampering was occurring” and focused on root cause investigation and access restriction strengthening.

  • Blocked access to phpMyAdmin
  • Only specific products became unpurchasable; we suspected data tampering but could not identify the tampered location before time ran out
  • In the post-game reveal, it turned out to be “product title tampering.” We had assumed “product information tampering (inventory or pricing)” and were heavily focused on investigating the database

Key Takeaways from the Reveal

  • Vulnerable WordPress installations may need to be “deleted” not just “disabled”
  • Some plugins have unimplemented disable functions, and data can still be passed from WordPress
  • WebDAV configuration and FTP server logs also require attention

Nobu Ueno was noted as having the most elegant approach:

  • In the first round, focus entirely on collecting logs
  • Matching the crawler’s 1-minute access cycle, change settings every minute to identify which configuration change causes purchases to fail

For general incident response, the following article is a useful reference:

Team Retrospective

Team Operations and Information Sharing

  • Clearly divided defensive roles for the response
  • Shared actions and findings in thread format after each set
  • Actively proposed suspicious points and solution ideas

AI Utilization

  • Submitted access and error logs to AI for advice and countermeasure suggestions

Things to Adopt in Future Development and Operations

  • Don’t include unnecessary libraries
  • Don’t use vulnerable versions
  • Thoroughly address end-of-life software
  • Enforce thorough permission management

Summary and Personal Learnings

  • Since I usually work with Splunk and k8s environments, I realized I had limited knowledge of Linux log locations and what to look for
  • The importance of checking a wide variety of logs
  • The need to systematically check tampered areas one by one without preconceptions
  • Don’t create unnecessary services or accounts