Cross-validation estimates how well a model may perform on unseen data by repeating training and validation across multiple splits. It provides more evidence than one convenient train-test split, but only when the splitting method reflects how the model will be used.
The central question is not “How many folds should I use?†It is “Which observations must remain separated to simulate the future?â€
K-fold cross-validation
K-fold cross-validation divides data into K parts. The model trains on K minus one folds and validates on the remaining fold, repeating until every fold has served as validation data. The final estimate usually reports the mean and variation of the metric.
Five or ten folds are common choices, but there is no perfect number. More folds increase computation and can change bias and variance. Choose a practical setting and report it.
Stratified K-fold
Stratification preserves approximately the same class ratio in every fold. It is useful for classification, particularly when one class is uncommon. Without stratification, a small validation fold might contain too few positive cases to produce a stable score.
Stratification does not solve imbalance by itself. It only makes class distribution more consistent across folds. Metrics, thresholds and sampling still require careful design.
Grouped cross-validation
Ordinary K-fold can leak information when several rows belong to the same person, machine, patient or account. A model may see one customer’s records during training and another record from that customer during validation.
Group-based splitting keeps every observation from a group in one fold. Use it for repeated measurements, multiple images from the same subject, branch-level data or any setting where related records would make validation easier than deployment.
Time-series validation
Random folds are usually inappropriate when predictions concern the future. Time-series validation trains on an earlier period and validates on a later period. An expanding window adds more history in each round; a rolling window keeps a fixed amount of recent history.
Leave a gap between training and validation when features or labels overlap near the boundary. Confirm that every feature would have been available at the simulated prediction time.
Keep preprocessing inside each fold
Imputation, scaling, feature selection, target encoding and resampling must be fitted only on the training portion of a fold. Fitting them once on the complete dataset leaks validation information.
Use a machine learning pipeline so every fold learns its own preprocessing parameters. The same rule applies to hyperparameter tuning: use nested validation or reserve a final untouched test set for an unbiased final check.
Read variation, not only the mean
Two models can have the same mean score but different stability. Report fold-level scores, standard deviation and confidence where appropriate. Investigate whether one time period, group or segment consistently performs poorly.
A large spread may indicate small data, distribution shifts, unreliable labels or a model that depends heavily on particular examples.
Match the metric to the problem
Regression may use MAE, RMSE or another loss linked to decision cost. Classification may require precision, recall, F1, log loss or precision-recall AUC. Use the same primary metric across model comparisons and define the direction of improvement.
For threshold-based metrics, decide whether the threshold is tuned inside each training process or fixed by policy. Avoid optimising it directly on the final test set.
Recommended project workflow
Start with a simple baseline. Select the split based on independence, groups and time. Place transformations inside a pipeline, compare models using identical folds and retain a final test set. Save the fold definitions or random seed so another person can reproduce the result.
The Machine Learning Training in Vizag connects evaluation theory with Python projects. Pair this guide with the imbalanced-data workflow and record every evaluation choice using machine learning experiment tracking.
Final takeaway
Cross-validation is a simulation of deployment. K-fold works for independent records, stratification stabilises class ratios, groups protect related entities and time-series splits respect chronology. The split design is part of the model, not an afterthought.