Holistic assessment is generally the tool of assessment used in almost all higher education settings for evaluating student writing samples produced in placement and exit examinations for academic writing classes. The major reasons for its use are its practicality and efficiency in terms of time in light of the reality that writing instructors teach multiple writing classes, and consequently, only a few instructor-raters are available to assess a large number of writing samples within a short period of time. Holistic evaluation has attained a great deal of face validity because of its wide use, and importantly, its reliability and validity have been often taken for granted by the writing instructors and program administrators. An in depth study of the tool per se, however, reveals that it has certain serious empirically ul1veri fied, unresolved and unstudied issues. For example, no study has so far shown what the goals of holistic evaluation are, how it conforms to construct validity, why rater reliability is to be preferred in lieu of instrument validity, whether holistic evaluation is a natural reading process, and so on. In fact, there is not even one full length in-depth research study that has attempted to show how holistic evaluation as a process works and under what cognitive, affective, and evaluative constraints do the raters process the student writing samples. An attempt has been made in this paper to highlight some of these major issues so that future research studies can focus on these concerns. Till then, it is recommended in this paper that holistic assessment is used as an assessment tool with great care and prudence since its issues may outweigh its benefits. The concluding section contains suggestions for future research.