To enhance the capability of sensing ride-hailing travel behavior and improve the accuracy of trip purpose inference, this study proposes an integrated framework consisting of POI type inference, multi-dimensional feature construction, and an XGBoost multi-class classification model based on the travel survey data from Yichun City. First, a POI size-weighted gravity model is developed by incorporating POI density, scale, and accessibility factors to improve the semantic identification of origins and destinations. Second, multi-dimensional behavioral features are constructed from temporal rhythms, spatial distance, and geographic semantics to characterize residents’ spatiotemporal travel patterns. Finally, the XGBoost model, optimized via Bayesian hyperparameter tuning, is applied for trip purpose classification. Experimental results show that the improved POI inference model achieves high accuracy across seven POI categories (up to 99.4%), effectively enhancing the quality of semantic inputs. The XGBoost model based on multi-dimensional features achieves an overall accuracy of 88.99%, significantly outperforming traditional POI-only methods (76.54%). Feature importance analysis indicates that destination POI type and departure/arrival time contribute most to model performance, which aligns with typical daily activity patterns. The proposed approach demonstrates strong interpretability and generalization ability and offers valuable technical support for ride-hailing dispatch optimization, urban travel demand analysis, and travel behavior modeling.