Skip to content

[BugFix][DataProcessor] Add validate_model_path to fail fast on bad model path or unreachable network#6713

Open
gongweibao wants to merge 3 commits intoPaddlePaddle:developfrom
gongweibao:fixmodelwarning
Open

[BugFix][DataProcessor] Add validate_model_path to fail fast on bad model path or unreachable network#6713
gongweibao wants to merge 3 commits intoPaddlePaddle:developfrom
gongweibao:fixmodelwarning

Conversation

@gongweibao
Copy link
Collaborator

Motivation

When model_name_or_path is incorrect or the network is unreachable, DataProcessor.__init__ hangs for 50s+ with zero output because from_pretrained silently retries HuggingFace Hub connections (10s timeout × 5 retries + exponential backoff). Users have no idea what is happening and assume the program is dead.

Modifications

  • Extract validate_model_path() into fastdeploy/input/utils.py: a lightweight pre-check that runs before any from_pretrained call. It checks whether the path is a local dir/file; if not, it warns the user and does a quick socket.create_connection probe (3s timeout) to detect network issues immediately.
  • Call validate_model_path() in both text_processor.py and v1/text_processor.py: one-line change in each DataProcessor.__init__, replacing the previous duplicated inline logic that had a bug (misidentified HF repo ids containing / as local paths).
  • Add unit tests in tests/input/test_validate_model_path.py: 7 test cases covering normal paths (local dir, local file), non-local path with reachable/unreachable network, boundary values (HF org/model name with /, empty string), and socket timeout.

Usage or Command

# Run unit tests
python -m pytest tests/input/test_validate_model_path.py -v

Accuracy Tests

Not applicable — this change only adds a pre-check warning before from_pretrained calls. It does not modify model forward code, kernels, or any inference logic.

Checklist

  • Add at least a tag in the PR title. ([BugFix], [DataProcessor])
  • Format your code, run pre-commit before commit.
  • Add unit tests. (7 cases in tests/input/test_validate_model_path.py)
  • Provide accuracy results. (N/A, no model output change)

@paddle-bot
Copy link

paddle-bot bot commented Mar 7, 2026

Thanks for your contribution!

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


gongweibao seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov-commenter
Copy link

codecov-commenter commented Mar 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (develop@aac1484). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #6713   +/-   ##
==========================================
  Coverage           ?   72.07%           
==========================================
  Files              ?      392           
  Lines              ?    53764           
  Branches           ?     8448           
==========================================
  Hits               ?    38749           
  Misses             ?    12224           
  Partials           ?     2791           
Flag Coverage Δ
GPU 72.07% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants