Phylogenies in short, are the most convenient way to describe the relationship between different species and are widely used in several fields of biology: comparative genomics, epidemiology, conservation biology, etc. However, most inferences drawn from phylogenies are accurate only if the reconstructed phylogeny itself is accurate. For a given reconstruction bias, robust phylogenies are preferred to non robust ones. We are concerned here with the loss of robustness induced by outliers. One way to mitigate this loss is to detect and remove outliers from the dataset. We advocate the use of empirical influence functions to detect influent characters and taxa, which are prone to be outliers, and their removal from the data set to build robust phylogenies. Three data sets (Zygomycetes, placental mammals, T-box gene family) show that maximum likelihood phylogenies are not robust and that removing as few as a handful of outliers can significantlyincrease the robustness of a tree, as measured by average bootstrap values.