by Elena McKee-Dabbs
Trying to identify portfolios that do not belong in a composite can be a difficult and time consuming chore. However, identifying these portfolios before performance is published or initial data is sent to Ashland Partners, can reduce the number of errors and improve the efficiency of the verification or examination. What follows are some simple mathematical tests that can be performed to help identify outliers and thus potential errors.
The most basic test to complete is to identify the two portfolios in the composite with the highest returns for the month. Compare the highest returning portfolio to the next highest returning portfolio to see if there is a large difference in the two returns. If there is, further analysis could be completed by reviewing the holdings in the portfolio compared to another portfolio in the composite. The same can be done for the lowest returning portfolio for the month in comparison to the next lowest returning portfolio. These portfolios may need to be reviewed because each performed better or worse than all of the other portfolios for the month and may have different holdings or allocations which are not in line with the composite.
A more mathematically meaningful approach would be to use the standard deviation. The typical approach is to determine the standard deviation of the underlying portfolio returns in a composite and then determine how many standard deviations will be used to identify potential errors. The assumption here is that the returns are normally distributed around the mean. This assumes that there are an equal amount of returns above and below the mean. Given a normal distribution, 99.7% of all values should fall within three standard deviations from the mean.
For example, take a composite made up of the following monthly portfolio level returns: 2.3%, 2.4%, 2.5%, 2.3%, 2.2%, 2.1%, 2.4%, 2.5%, 2.3%, 2.3%, 2.3%, 2.3% 10%, 40%. The mean return for this set of returns is 5.8%. The standard deviation is calculated as 10.1%. Using our criteria noted above of three standard deviations, this would indicate that 99.7% of all returns should lie within -24.4% and 36.0%. This test would flag 40% as an outlier, but ignore the 10%. This is because the returns are heavily skewed to the right. If it is determined that the portfolio with the 40% return does not belong in the composite, the standard deviation would then be re-calculated. Removing the 40% return, the average and standard deviation is 3.0% and 2.1%, respectively. Now we would expect 99.7% of all returns to lie within-3.4% and 9.3%. This would now flag the 10% return as an outlier. One final calculation on the remaining data set would reveal a mean of 2.3% and a standard deviation of 0.1%, which does not indicate there are any remaining outliers.
The problem with the method described above, is that it assumes a normal distribution, and if there are outliers, the data set is not normally distributed. To accommodate for this, another test that could be completed would be to review the historical average of monthly standard deviations. In the case above, if this composite typically has a monthly standard deviation between 0.1% and 0.4%, with an average of 0.2%, a standard deviation of 5.8% would be unusual and would warrant deeper investigation. An advisor could use the average standard deviation for the composite to determine outliers for the current month. In this way, both the 10% and 40% return outliers in the previous example would be flagged initially.
Ashland Partners uses several methods for determining outliers and deciding whether to test a portfolio for placement testing. Reviewing your composites for monthly outliers is a best practice. How often, and what type of review depends on the type of composite, what controls are in place, and the resources that are dedicated to composite maintenance.