In R ggplot2, include stat_ecdf() endpoints (0,0) and (1,1)

Question

I'm trying to use stat_ecdf() to plot cumulative successes as a function of a rank score created by a predictive model.

#libraries
require(ggplot2)
require(scales)

# fake data for reproducibility
set.seed(123)
n <- 200
df <- data.frame(model_score= rexp(n=n,rate=1:n),
                 obs_set= sample(c("training","validation"),n,replace=TRUE))
df$model_rank <- rank(df$model_score)/n
df$target_outcome <- rbinom(n,1,1-df$model_rank)

# Plot Gain Chart using stat_ecdf()
ggplot(subset(df,target_outcome==1),aes(x = model_rank)) + 
  stat_ecdf(aes(colour = obs_set), size=1) + 
  scale_x_continuous(limits=c(0,1), labels=percent,breaks=seq(0,1,.1)) +
  xlab("Model Percentile") + ylab("Percent of Target Outcome") +
  scale_y_continuous(limits=c(0,1), labels=percent) +
  geom_segment(aes(x=0,y=0,xend=1,yend=1), 
               colour = "gray", linetype="longdash", size=1) +
  ggtitle("Gain Chart")

enter image description here

All I want to do is force the ECDF to start at (0,0) and end at (1,1) so that there are no gaps at the beginning or end of the curve. If possible, I'd like to do it within the syntax of ggplot2, but I'd settle for a clever workaround.

@Henrik this is NOT a duplicate of this question, because I have already defined my limits with scale_x_ and _y_continuous(), and adding expand_limits() doesn't do anything. It is not the origin of the PLOT but the endpoints of the stat_ecdf() that need fixed.

@Henrik -- read the question. Look at my plot. Clearly the grid includes (0,0) and (1,1) already, points which are included on the `geom_segment()` nonetheless, so it is *not* a question of setting the limits of the plot object. The question has to do with the domain of the `stat_ecdf()` object as stated in the title of my question. Look at what you consider to be a duplicate post. It is not related at all. — C8H10N4O2, Feb 19 '15 at 15:25
Can you tell me how did you do for having percent on y-axis ? — Benmoshe, Apr 22 '15 at 13:01
@MostafaRifi `+ scale_y_continuous(labels=percent)` and don't forget `library(scales)` — C8H10N4O2, Apr 22 '15 at 14:48

score 1 · Accepted Answer · answered Aug 05 '15 at 14:54

Unfortunately, the definition of stat_ecdf gives no wiggle room here; it determines the endpoints internally.

There is a somewhat advanced solution. With the latest version of ggplot2 (devtools::install_github("hadley/ggplot2")), the extensibility is improved, to the point where it is possible to override this behavior, but not without some boilerplate.

stat_ecdf2 <- function(mapping = NULL, data = NULL, geom = "step",
                      position = "identity", n = NULL, show.legend = NA,
                      inherit.aes = TRUE, minval=NULL, maxval=NULL,...) {
  layer(
    data = data,
    mapping = mapping,
    stat = StatEcdf2,
    geom = geom,
    position = position,
    show.legend = show.legend,
    inherit.aes = inherit.aes,
    stat_params = list(n = n, minval=minval,maxval=maxval),
    params = list(...)
  )
}


StatEcdf2 <- ggproto("StatEcdf2", StatEcdf,
  calculate = function(data, scales, n = NULL, minval=NULL, maxval=NULL, ...) {
    df <- StatEcdf$calculate(data, scales, n, ...)
    if (!is.null(minval)) { df$x[1] <- minval }
    if (!is.null(maxval)) { df$x[length(df$x)] <- maxval }
    df
  }
)

Now, stat_ecdf2 will behave the same as stat_ecdf, but with an optional minval and maxval parameter. So this will do the trick:

ggplot(subset(df,target_outcome==1),aes(x = model_rank)) +
  stat_ecdf2(aes(colour = obs_set), size=1, minval=0, maxval=1) +
  scale_x_continuous(limits=c(0,1), labels=percent,breaks=seq(0,1,.1)) +
  xlab("Model Percentile") + ylab("Percent of Target Outcome") +
  scale_y_continuous(limits=c(0,1), labels=percent) +
  geom_segment(aes(x=0,y=0,xend=1,yend=1),
               colour = "gray", linetype="longdash", size=1) +
  ggtitle("Gain Chart")

The big caveat here is that I don't know if the current extensibility model will be supported in the future; it has changed several times in the past, and the change to use "ggproto" is recent -- like July 15th 2015 recent.

As a plus, this gave me a chance to really dig into ggplot's internals, which is something that I've been meaning to do for a while.

I learned a lot from this answer -- thanks. Much better than my workaround, which created dummy data points (at the risk of distorting the ecdf, at least on smaller data sets). This creates dummy points on the plot itself. — C8H10N4O2, Aug 05 '15 at 15:10

In R ggplot2, include stat_ecdf() endpoints (0,0) and (1,1)

1 Answers1

Linked