You cannot simply depart on an expired document. The official answer gives two routes: accept the penalty and complete the documents at the local exit-entry authority before leaving, or, for urgent departure, accept the penalty at the exit port and then depart.
Nationwide (all cities)
Departure requires inspection and approval
Article 27 of the Exit and Entry Administration Law requires a foreigner leaving China to present their passport or other exit-entry document to the border inspection authority and to complete the prescribed procedures before being permitted to depart. Article 37 requires departure before the stated stay or residence period expires. Once the document has expired, the overstay has to be dealt with first.
Route 1: settle it locally first
The Huaibei municipal government answer states that a foreigner who has overstayed should go to the local public security exit-entry administration department to accept the penalty and complete the relevant documents before leaving China.
Route 2: penalty at the port
The same official answer states that where special circumstances require urgent departure, the foreigner may also accept the penalty from the border inspection authority at the exit port and then depart. Article 69 of the NIA border inspection provisions likewise allows the border inspection authority to collect a fine on the spot when paying at a designated bank or through the electronic payment system at the port is genuinely difficult.
What the law does not say about unpaid fines
Article 28 of the Exit and Entry Administration Law lists four grounds on which a foreigner may not leave China: an uncompleted criminal sentence or being a defendant or suspect in a criminal case; an unresolved civil case where a court has decided to bar departure; unpaid labour remuneration where a decision to bar departure has been made by the relevant State Council department or provincial government; and other grounds under laws and administrative regulations. An unpaid administrative fine is not among them. The official texts do not state that "unpaid fines always block boarding".
Appeal does not suspend enforcement
Article 68 of the NIA border inspection provisions states that applying for administrative reconsideration or filing an administrative lawsuit does not suspend enforcement of the administrative decision, unless the law provides otherwise.