| Title: | Bayesian False-Positive Occupancy Models with Calibration Checks |
|---|---|
| Description: | Fits single-season occupancy models that correct for false positives from automated detectors, such as acoustic classifiers, using the multiple-methods (site-confirmation) design of Miller et al. (2011) <doi:10.1890/10-1396.1>, in which a subset of detections is verified. Models are written in 'Stan' and fitted through 'cmdstanr'. The false-positive rate can be constant or depend on site covariates, and the true-detection process accepts observation covariates and a recorder random effect. The package also simulates data under homogeneous, unstructured, and structured false-positive regimes and runs simulation-based calibration checks, so users can ask whether the correction keeps nominal interval coverage for their own design and how much verification it needs. |
| Authors: | Diogo B. Provete [aut, cre] (ORCID: <https://orcid.org/0000-0002-0097-0651>), Tessa Rhinehart [aut] (ORCID: <https://orcid.org/0000-0002-4352-3464>), Liliana Piatti [aut] (ORCID: <https://orcid.org/0000-0001-6324-1723>), Larissa Sayuri Moreira Sugai [aut] (ORCID: <https://orcid.org/0000-0002-6248-291X>) |
| Maintainer: | Diogo B. Provete <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 1.3.0 |
| Built: | 2026-10-20 05:26:59 UTC |
| Source: | https://github.com/diogoprov/crywolf |
Detections of the Common Pauraque (Nyctidromus albicollis) by two automated classifiers in one-minute recordings made at seven sites in the Pantanal wetlands and adjacent Cerrado, Mato Grosso do Sul, Brazil, between November 2023 and February 2025, with an independent set of recordings annotated by ear. The occupancy unit is the site-night, the occasion is the one-minute recording, and the annotated recordings are the verified method of the multiple-methods design.
curiangocuriango
A list with three data frames:
minutesOne row per one-minute recording (97,806 rows):
site_night (site-night identifier, the occupancy unit), site
(recorder site, one of seven), time (start of the recording),
conf_custom and conf_birdnet (confidence scores of a custom-trained
classifier and of the BirdNET global model v2.4, 0 when the classifier
returned nothing), and the acoustic indices bio (Bioacoustic Index) and
aci (Acoustic Complexity Index).
nightsOne row per site-night (1,753 rows): site_night, site,
night (date of the evening), and the nightly means of temperature
(temp, °C) and relative humidity (rh, %), and the fraction of the
lunar disc illuminated (moon).
verifiedOne row per annotated recording (1,199 rows):
site_night, site, time, and y, 1 if any 3-s segment of the
recording held the species.
The annotated recordings are an independent ground-truth test set of about
1% of the recordings, chosen in proportion to the number of recordings at
each site, and annotated without reference to the classifiers (Piatti et
al., 2026). They can therefore serve as the verified method of
fp_data(). A detection by a classifier is a confidence score at or above a
threshold; Piatti et al. (2026) used 0.40 (custom) and 0.10 (BirdNET) to
maximize F1, and 0.95 for both to maximize precision.
The source table has one to 20 rows (detection windows) per recording,
and a classifier's score is non-zero only in the windows where it detected
the species. This dataset keeps one row per recording, with the maximum
score over its windows, so a recording is a detection when any of its
windows reaches the threshold. The script
that builds it from the public deposit is data-raw/curiango.R in the
package repository.
Piatti, L., de Sousa, D. L. H., Oliveira, B. S., de Melo, A. V., Sugai, L. S. M., & Provete, D. B. (2026). Data and code for: Methodological choices influence ecological inference in passive acoustic monitoring of a Neotropical nightjar. Zenodo. doi:10.5281/zenodo.21795898. Licence CC BY 4.0.
Piatti, L., de Sousa, D. L. H., Oliveira, B. S., de Melo, A. V., Sugai, L. S. M., & Provete, D. B. (2026). Methodological choices influence ecological inference in passive acoustic monitoring of a Neotropical nightjar. Journal of Avian Biology, in press.
str(curiango, max.level = 1) # detections by the custom classifier at the MaxF1 threshold mean(curiango$minutes$conf_custom >= 0.40)str(curiango, max.level = 1) # detections by the custom classifier at the MaxF1 threshold mean(curiango$minutes$conf_custom >= 0.40)
Simulates many datasets from a design with simulate_fp(), fits one or more
false-positive models to each with fp_occu(), and scores how often the
credible intervals contain the true values and how biased the posterior
means are. Use it before collecting data, to ask whether the correction
will be calibrated for the number of sites, occasions, and verified sites
you can afford, and whether a constant false-positive rate is enough under
the confusion you suspect.
fp_calibrate( design, models = list(constant = ~1), occ = ~x, det = ~1, reps = 50, prob = 0.95, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )fp_calibrate( design, models = list(constant = ~1), occ = ~x, det = ~1, reps = 50, prob = 0.95, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )
design |
A data frame (or list) whose columns are arguments of
|
models |
Named list of formulas for the false-positive rate, one model
per element, as in the |
occ, det
|
Formulas for occupancy and true detection, shared by all
models. Simulated data have the occupancy covariate |
reps |
Number of replicate datasets per cell. Each costs one Stan fit per model. Fifty replicates give a Monte Carlo standard error of about 0.03 for a coverage of 0.95. |
prob |
Width of the credible intervals scored. |
base_seed |
Integer. Replicate |
parallel |
Logical. Fit replicates in parallel with 'furrr'. Set the
plan first, e.g. |
fit_args |
List of further arguments for |
The scored quantities are the occupancy intercept and slope
(occ_(Intercept), occ_x), the false-positive rate averaged over sites
(p10_bar, against the realized mean rate), the detection probability of
the verified method (p_verified), and, for models that include the
confusing gradient w, its effect on the false-positive rate (fp_w).
A tibble of class fp_calibration, one row per replicate, model,
and scored quantity, with the truth, the posterior mean and interval,
whether the interval covered the truth, and sampler diagnostics. Use
base::summary() for coverage, its Monte Carlo standard error, bias, and RMSE
per cell and model.
fp_verification_curve(), simulate_fp()
## Not run: design <- data.frame(p10 = 0.05, conf_effect = c(0, 1.5), conf_cor = 0.7) cal <- fp_calibrate(design, models = list(constant = ~ 1, fp_on_x = ~ x), occ = ~ x, reps = 50) summary(cal) ## End(Not run)## Not run: design <- data.frame(p10 = 0.05, conf_effect = c(0, 1.5), conf_cor = 0.7) cal <- fp_calibrate(design, models = list(constant = ~ 1, fp_on_x = ~ x), occ = ~ x, reps = 50) summary(cal) ## End(Not run)
Asks whether the credible intervals of a model would cover the truth in
data like the ones it was fitted to. The function simulates datasets from
the fit with simulate.fp_occu(), so that they keep the design of the
real data (sites, covariates, occasions, verification, recorders) and
follow the fitted model, refits each candidate model to every dataset,
and scores the coverage and bias of their intervals, as fp_calibrate()
does for designed simulations.
fp_calibrate_fit( fit, models = list(fitted = fit$formulas$fp), reps = 50, truth = c("mean", "posterior"), prob = 0.95, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )fp_calibrate_fit( fit, models = list(fitted = fit$formulas$fp), reps = 50, truth = c("mean", "posterior"), prob = 0.95, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )
fit |
An |
models |
Named list of false-positive formulas (or |
reps |
Number of simulated datasets. |
truth |
|
prob |
Width of the credible intervals. |
base_seed |
Seeds are |
parallel, fit_args
|
As in |
The scored quantities are the occupancy coefficients, the detection
coefficients, the intercept of the verified method, the mean
false-positive rate over sites (p10_bar, when both the generating and
the refitted model have false positives), and the slopes of the
false-positive rate in the refitted model. A slope that is not in the
generating model has a true value of zero.
The most useful comparison pits the model you fitted against a simpler
one. For example, if a model with fp = ~ temp + rh estimated clear
effects of the covariates on the false-positive rate, refitting
fp = ~ 1 to data simulated from it shows how much a constant rate would
bias the occupancy effects at your sample size and verification effort.
This answers the question for the fitted structure only: a dependence of
the false-positive rate on something you did not measure is not part of
the generating model.
Each replicate refits every model, so the cost is that of reps times
the number of models fits of the size of the real data. Use parallel,
and threads_per_chain in fit_args for large datasets.
An object of class fp_calibration, as from fp_calibrate(),
with a single design cell.
fp_calibrate(), simulate.fp_occu()
## Not run: sim <- simulate_fp(nsite = 300, nocc = 20, conf_effect = 1.5, conf_cor = 0.7, seed = 1) m_x <- fp_occu(sim$data, occ = ~ x, fp = ~ x, random_recorder = FALSE) cal <- fp_calibrate_fit(m_x, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 20) summary(cal) plot(cal, parameter = "occ_x") ## End(Not run)## Not run: sim <- simulate_fp(nsite = 300, nocc = 20, conf_effect = 1.5, conf_cor = 0.7, seed = 1) m_x <- fp_occu(sim$data, occ = ~ x, fp = ~ x, random_recorder = FALSE) cal <- fp_calibrate_fit(m_x, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 20) summary(cal) plot(cal, parameter = "occ_x") ## End(Not run)
Compiles the false-positive occupancy model the first time it is needed and
keeps the executable in the user cache directory
(tools::R_user_dir("crywolf", "cache")), so later sessions load it
without recompiling. fp_occu() calls this function itself; call it
directly to compile ahead of time, for example before launching parallel
workers in fp_calibrate().
fp_compile(force = FALSE, quiet = TRUE)fp_compile(force = FALSE, quiet = TRUE)
force |
Logical. Recompile even if a current executable exists. |
quiet |
Logical. Suppress compiler output. |
The model is compiled with threading enabled (stan_threads), so that
fp_occu() can split the likelihood of each chain among several threads
(argument threads_per_chain).
A CmdStanModel object (invisibly).
## Not run: fp_compile() ## End(Not run)## Not run: fp_compile() ## End(Not run)
Collects the automated detections, the verified detections, and the site
table into one object that fp_occu() can fit. All tables are in long
format, one row per occasion, which accommodates unequal numbers of
occasions per site without padding. Use fp_data_wide() if your data are
detection matrices, as in 'unmarked'.
fp_data( auto, sites, verified = NULL, site = "site", y = "y", recorder = NULL, confirmed = NULL )fp_data( auto, sites, verified = NULL, site = "site", y = "y", recorder = NULL, confirmed = NULL )
auto |
Data frame of the automated method (the one that can return false positives), one row per site and occasion. It needs a site column and a 0/1 detection column, and may hold observation covariates for the true-detection process and a recorder (or observer) identifier. |
sites |
Data frame with one row per site: the site identifier plus any
site covariates used in the occupancy ( |
verified |
Optional data frame of the verified method (detections
checked by an expert and therefore free of false positives), one row per
site and occasion, with the same site and detection columns. Without it
the false-positive rate is identified only by the priors, and
|
site, y
|
Names of the site and detection columns, shared by all tables. |
recorder |
Optional name of a column of |
confirmed |
Optional names of two columns of |
An object of class fp_data: a list with the three tables (rows
with a missing detection dropped), the column names, and the vector
known_present, which flags sites with at least one verified detection
or confirmed true observation.
Chambert, T., Miller, D. A. W., & Nichols, J. D. (2015). Modeling false positive detections in species occurrence data under different study designs. Ecology, 96, 332–339. doi:10.1890/14-1507.1
fp_data_wide(), fp_occu(), simulate_fp()
sim <- simulate_fp(nsite = 50, nocc = 5, seed = 1) sim$data # the same structure built by hand d <- fp_data(auto = sim$data$auto, sites = sim$data$sites, verified = sim$data$verified, recorder = "recorder") dsim <- simulate_fp(nsite = 50, nocc = 5, seed = 1) sim$data # the same structure built by hand d <- fp_data(auto = sim$data$auto, sites = sim$data$sites, verified = sim$data$verified, recorder = "recorder") d
Convenience wrapper around fp_data() for data stored as site-by-occasion
matrices, the layout used by 'unmarked'. Missing values (NA) mark
occasions that were not sampled, so sites may differ in effort.
fp_data_wide( y_auto, y_verified = NULL, site_covs = NULL, obs_covs = NULL, recorder = NULL )fp_data_wide( y_auto, y_verified = NULL, site_covs = NULL, obs_covs = NULL, recorder = NULL )
y_auto |
Matrix of automated detections (sites in rows, occasions in
columns), coded 0/1/ |
y_verified |
Optional matrix of verified detections with the same
number of rows; unverified sites are rows of |
site_covs |
Optional data frame of site covariates, one row per site,
in the same order as the rows of |
obs_covs |
Optional named list of matrices, each the same size as
|
recorder |
Optional vector, one value per site, identifying the recorder or observer. |
An object of class fp_data.
set.seed(1) ya <- matrix(rbinom(40, 1, 0.3), 10, 4) yv <- matrix(NA, 10, 2); yv[1:3, ] <- rbinom(6, 1, 0.5) fp_data_wide(ya, yv, site_covs = data.frame(x = rnorm(10)))set.seed(1) ya <- matrix(rbinom(40, 1, 0.3), 10, 4) yv <- matrix(NA, 10, 2); yv[1:3, ] <- rbinom(6, 1, 0.5) fp_data_wide(ya, yv, site_covs = data.frame(x = rnorm(10)))
Fits the multiple-methods false-positive occupancy model of Miller et al. (2011) in Stan. Detections from an automated method may be false positives, whereas detections from a verified subset are certain. The occupancy probability depends on site covariates, true detection on observation (or site) covariates and an optional recorder random intercept, and the false-positive rate is either constant or a function of site covariates.
fp_occu( data, occ = ~1, det = ~1, fp = ~1, random_recorder = !is.null(data$recorder), random_fp = FALSE, priors = fp_priors(), chains = 4, parallel_chains = chains, threads_per_chain = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95, seed = NULL, refresh = NULL, ... )fp_occu( data, occ = ~1, det = ~1, fp = ~1, random_recorder = !is.null(data$recorder), random_fp = FALSE, priors = fp_priors(), chains = 4, parallel_chains = chains, threads_per_chain = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95, seed = NULL, refresh = NULL, ... )
data |
An |
occ |
One-sided formula for the logit of occupancy, using columns of
|
det |
One-sided formula for the logit of the true-detection probability
of the automated method. It may use columns of |
fp |
One-sided formula for the logit of the false-positive rate, using
columns of |
random_recorder |
Logical. Add a recorder random intercept to true
detection? Defaults to |
random_fp |
Logical. Add a site random intercept to the logit of the
false-positive rate, for rates that vary among sites in ways the
covariates in |
priors |
Prior settings from |
chains, iter_warmup, iter_sampling, adapt_delta, seed
|
Passed to the
|
parallel_chains |
Number of chains run in parallel. |
threads_per_chain |
Number of threads each chain uses to compute the
likelihood, which is summed over sites in parallel with Stan's
|
refresh |
How often each chain reports its progress, in iterations.
The default, |
... |
Further arguments passed to |
Identifiability. The intercept of the false-positive rate equals the
intercept of true detection minus a positive gap, , so that at the reference values of the covariates (all zero) a
true detection is more likely than a false positive. This removes the label
switching between the occupied and unoccupied states. Centre the
covariates, so that the reference is an average site; simulate_fp()
already returns them standardized.
Verified data. The verified detections anchor the false-positive rate. With none, the model still runs, but the rate is informed only by the priors and the gap, and the fit should not be trusted.
Observation-confirmation design. When data has confirmed occasions
(argument confirmed of fp_data()), the model follows the
observation-confirmation design of Chambert et al. (2015): an expert checks
every observation of some automated occasions and finds only false ones,
only true ones, both, or none. False observations can then occur at
occupied sites too, and det and fp model and , the
probabilities of at least one true and at least one false observation on an
occasion; a site-level detection at an occupied site has probability
, and p11_ref reports it at the
reference. The confirmed occasions identify directly, so verified
data are optional; they can be combined with this design. The gap still
applies, so that at the reference.
Site random effect on the false-positive rate. With
random_fp = TRUE the logit of the false-positive rate of site
gains a term , reported as
sd_fp_site, with a half-Normal prior (fp_priors(fp_site_sd = )).
In simulations with 300 sites, 20 automated occasions, and 30% of sites
verified, it restored the coverage of the mean false-positive rate
(p10_bar) when the rate varied among sites at random (from 0.00 and 0.55
with a constant rate to 0.95) and cost nothing when the rate was
homogeneous. Use it with care:
It adds one parameter per site, and fits took 6 to 9 times longer.
It mixes less well: in those simulations the largest R-hat exceeded 1.01
in a quarter to three quarters of the fits. Check the diagnostics, and
run longer chains (iter_warmup, iter_sampling) when they are poor.
Each site has a single latent state, so a site effect on false positives is told apart from occupancy mainly by the verified data; with few occasions or little verification it may be weakly identified.
It does not fix false positives that follow an environmental gradient.
Those bias the occupancy effects, and the remedy is to put the
occupancy covariates, and any measured correlate of the confusion, in
fp (see below). Prefer covariates, and add the random effect for the
variation that remains.
Predictions with predict(process = "fp") are for a site with
, a typical site.
When to model the false-positive rate. In the simulations behind this
package, a constant rate gave calibrated intervals for the occupancy slope
when false positives were homogeneous; random variation of the rate among
sites made the estimate of the rate unreliable but largely spared the slope.
When the rate tracked an environmental gradient correlated with the
occupancy covariate and the confusion was strong, the slope was biased and
verifying more sites did not fix it. Placing the occupancy covariate in fp
recovered part of the calibration, and placing the gradient itself
recovered all of it. Use fp_calibrate() to check your own design.
Parameter names. Coefficients are labelled by process and column of the
design matrix: occ_* (occupancy), det_* (true detection), fp_* (false
positive, where fp_(Intercept) is ), plus fp_gap
(), sd_recorder, p_verified (detection probability of the
verified method), p11_ref and p10_ref (true- and false-positive
probabilities at the reference), and p10_bar (false-positive probability
averaged over the sites).
An object of class fp_occu, with methods for base::print(),
base::summary(), stats::coef(), stats::predict(), loo::loo(), and
posterior::as_draws_df(). The 'cmdstanr' fit is in element fit.
Chambert, T., Miller, D. A. W., & Nichols, J. D. (2015). Modeling false positive detections in species occurrence data under different study designs. Ecology, 96, 332–339. doi:10.1890/14-1507.1
Miller, D. A., Nichols, J. D., McClintock, B. T., Grant, E. H. C., Bailey, L. L., & Weir, L. A. (2011). Improving occupancy estimation when two types of observational error occur: non-detection and species misidentification. Ecology, 92, 1422–1428. doi:10.1890/10-1396.1
fp_data(), fp_priors(), fp_calibrate()
## Not run: sim <- simulate_fp(nsite = 300, nocc = 20, conf_effect = 1.5, conf_cor = 0.7, seed = 42) # constant false-positive rate (the simulated recorders have no effect, # so the recorder term is left out) m0 <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) # false-positive rate that tracks the occupancy covariate m1 <- fp_occu(sim$data, occ = ~ x, fp = ~ x, random_recorder = FALSE) coef(m0)["occ_x"]; coef(m1)["occ_x"] # truth: -0.8 loo::loo_compare(loo(m0), loo(m1)) ## End(Not run)## Not run: sim <- simulate_fp(nsite = 300, nocc = 20, conf_effect = 1.5, conf_cor = 0.7, seed = 42) # constant false-positive rate (the simulated recorders have no effect, # so the recorder term is left out) m0 <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) # false-positive rate that tracks the occupancy covariate m1 <- fp_occu(sim$data, occ = ~ x, fp = ~ x, random_recorder = FALSE) coef(m0)["occ_x"]; coef(m1)["occ_x"] # truth: -0.8 loo::loo_compare(loo(m0), loo(m1)) ## End(Not run)
Summaries, point estimates, predictions, and model comparison for objects
returned by fp_occu().
## S3 method for class 'fp_occu' print(x, digits = 3, ...) ## S3 method for class 'fp_occu' summary(object, prob = 0.95, ...) ## S3 method for class 'summary.fp_occu' print(x, digits = 3, ...) ## S3 method for class 'fp_occu' coef(object, type = c("all", "occ", "det", "fp"), ...) ## S3 method for class 'fp_occu' predict( object, newdata, process = c("occ", "fp"), prob = 0.95, summary = TRUE, ... ) ## S3 method for class 'fp_occu' loo(x, ...) ## S3 method for class 'fp_occu' as_draws_df(x, ...)## S3 method for class 'fp_occu' print(x, digits = 3, ...) ## S3 method for class 'fp_occu' summary(object, prob = 0.95, ...) ## S3 method for class 'summary.fp_occu' print(x, digits = 3, ...) ## S3 method for class 'fp_occu' coef(object, type = c("all", "occ", "det", "fp"), ...) ## S3 method for class 'fp_occu' predict( object, newdata, process = c("occ", "fp"), prob = 0.95, summary = TRUE, ... ) ## S3 method for class 'fp_occu' loo(x, ...) ## S3 method for class 'fp_occu' as_draws_df(x, ...)
x, object
|
An |
digits |
Number of digits to print. |
... |
Unused, or passed on (see each method). |
prob |
Width of the equal-tailed credible interval. |
type |
Which coefficients to return: |
newdata |
Data frame with the covariates of the requested process. If missing, the sites used in the fit. |
process |
|
summary |
Logical. Return a summary per row ( |
summary() returns an object of class summary.fp_occu whose
element table is a tibble with the posterior mean, standard deviation,
credible interval, R-hat, and effective sample sizes of every parameter.
coef() returns a named vector of posterior means.
predict() returns a data frame with the posterior mean and
credible interval of the probability for each row of newdata, or a
matrix of draws.
loo() returns a psis_loo object from 'loo'. The pointwise
log-likelihood is computed per site, with the latent state marginalized,
so the comparison is leave-one-site-out.
as_draws_df() returns the posterior draws with the readable
parameter names.
## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) m # posterior summary and diagnostics summary(m, prob = 0.9)$table coef(m, "occ") predict(m, newdata = data.frame(x = c(-1, 0, 1)), process = "occ") loo(m) draws <- as_draws_df(m) ## End(Not run)## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) m # posterior summary and diagnostics summary(m, prob = 0.9)$table coef(m, "occ") predict(m, newdata = data.frame(x = c(-1, 0, 1)), process = "occ") loo(m) draws <- as_draws_df(m) ## End(Not run)
Replicates the detection data from the posterior of an fp_occu() fit and
compares observed and replicated data with a discrepancy statistic, giving a
Bayesian p-value. Detections are summed per site, so the check asks whether
the model reproduces how many detections each site gets.
fp_ppc( object, method = c("automated", "verified"), fit_stat = c("freeman-tukey", "chi-squared"), ndraws = 1000, seed = NULL )fp_ppc( object, method = c("automated", "verified"), fit_stat = c("freeman-tukey", "chi-squared"), ndraws = 1000, seed = NULL )
object |
An |
method |
Which data to check: |
fit_stat |
Discrepancy statistic: |
ndraws |
Number of posterior draws to use, sampled without replacement.
|
seed |
Optional integer seed for the subsampling and replication. |
The latent occupancy state is marginalized in the Stan model, so for each
posterior draw is first drawn from its conditional posterior given
the data and the parameters,
which is 1 at sites with a verified detection. Given , automated
detections are replicated with probability and verified detections with probability . The
discrepancy is computed for the observed and the replicated data with the
same and parameters, and the Bayesian p-value is the proportion of
draws in which the replicated discrepancy exceeds the observed one. Values
near 0 or 1 indicate lack of fit; values near 0.5 indicate that the
observed data are typical of the fitted model.
Under the observation-confirmation design, the conditional probability of
also uses the confirmed occasions, and "automated" checks the
occasions that were not confirmed.
A p-value near 0.5 does not show that the false-positive process is
correctly specified: a check of detection counts can have little power
against a false-positive rate that tracks an unmeasured gradient. Use
fp_calibrate() to ask whether the intervals of a given model are
calibrated for a design.
The elements y and yrep follow the conventions of the 'bayesplot'
package, so its ppc_*() functions can be used directly, for example
bayesplot::ppc_bars(ppc$y, ppc$yrep).
An object of class fp_ppc, a list with y (observed detections
per site), yrep (matrix of replicated detections, draws in rows and
sites in columns), fit_obs and fit_rep (discrepancy of the observed and
replicated data, one value per draw), p_value, site (the site
identifiers of the columns), method, and fit_stat.
## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) ppc <- fp_ppc(m, ndraws = 500, seed = 1) ppc bayesplot::ppc_bars(ppc$y, ppc$yrep) ## End(Not run)## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) ppc <- fp_ppc(m, ndraws = 500, seed = 1) ppc bayesplot::ppc_bars(ppc$y, ppc$yrep) ## End(Not run)
Normal priors on the logit scale. The defaults are weakly informative for standardized covariates.
fp_priors( intercept_sd = 1.5, slope_sd = 2.5, gap_mean = 2, gap_sd = 1.5, recorder_sd = 2, fp_site_sd = 1 )fp_priors( intercept_sd = 1.5, slope_sd = 2.5, gap_mean = 2, gap_sd = 1.5, recorder_sd = 2, fp_site_sd = 1 )
intercept_sd |
Standard deviation of the Normal(0, sd) priors on the intercepts of occupancy, true detection, and verified detection. |
slope_sd |
Standard deviation of the Normal(0, sd) priors on all slopes, including those of the false-positive rate. |
gap_mean, gap_sd
|
Mean and standard deviation of the Normal prior, truncated at zero, on the gap between the true-detection and false-positive intercepts. The default, Normal(2, 1.5), puts the false-positive rate at the reference roughly an order of magnitude below true detection, while leaving room for the data. |
recorder_sd |
Scale of the half-Normal(0, sd) prior on the standard deviation of the recorder random intercept. |
fp_site_sd |
Scale of the half-Normal(0, sd) prior on the standard
deviation of the site random intercept on the false-positive rate
( |
A list of class fp_priors.
fp_priors() fp_priors(slope_sd = 1)fp_priors() fp_priors(slope_sd = 1)
Path to the Stan program
fp_stan_file()fp_stan_file()
The path to the Stan file shipped with the package, for users who want to read the model or modify a copy of it.
fp_stan_file()fp_stan_file()
Runs fp_calibrate() over a grid of verification fractions and prints the
coverage of the occupancy slope against the fraction of sites verified. If
the constant-rate model does not approach nominal coverage as verification
grows, the false positives are probably structured, and the remedy is to
model the false-positive rate (for instance on the occupancy covariate),
not to verify more.
fp_verification_curve( ..., frac_verified = c(0.1, 0.3, 0.5, 0.7, 0.9), models = list(constant = ~1), occ = ~x, det = ~1, reps = 50, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )fp_verification_curve( ..., frac_verified = c(0.1, 0.3, 0.5, 0.7, 0.9), models = list(constant = ~1), occ = ~x, det = ~1, reps = 50, base_seed = 1L, parallel = FALSE, fit_args = list(chains = 4, parallel_chains = 1, iter_warmup = 1000, iter_sampling = 1000, adapt_delta = 0.95) )
... |
Arguments of |
frac_verified |
Fractions of sites verified to sweep. |
models, occ, det, reps, base_seed, parallel, fit_args
|
As in
|
The fp_calibration object (invisibly), after printing the curve.
## Not run: fp_verification_curve(p10 = 0.05, conf_effect = 1.5, conf_cor = 0.7, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 50) ## End(Not run)## Not run: fp_verification_curve(p10 = 0.05, conf_effect = 1.5, conf_cor = 0.7, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 50) ## End(Not run)
Plots the coverage (or bias) of one scored quantity against the column of the design that varies, one colour per model. Coverage is shown with an interval of two Monte Carlo standard errors and a dashed line at the nominal level; bias, with a dashed line at zero.
## S3 method for class 'fp_calibration' plot(x, parameter = "occ_x", type = c("coverage", "bias"), by = NULL, ...)## S3 method for class 'fp_calibration' plot(x, parameter = "occ_x", type = c("coverage", "bias"), by = NULL, ...)
x |
An |
parameter |
Scored quantity to plot, by default the occupancy slope
( |
type |
|
by |
Name of the design column for the horizontal axis. By default, the first design column that varies among cells; further varying columns become panels. With no varying column, the models are on the axis. |
... |
Unused. |
A 'ggplot2' object, which can be modified with the usual +
syntax.
## Not run: cal <- fp_verification_curve(p10 = 0.05, conf_effect = 1.5, conf_cor = 0.7, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 50) plot(cal) plot(cal, type = "bias") ## End(Not run)## Not run: cal <- fp_verification_curve(p10 = 0.05, conf_effect = 1.5, conf_cor = 0.7, models = list(constant = ~ 1, fp_on_x = ~ x), reps = 50) plot(cal) plot(cal, type = "bias") ## End(Not run)
Generates one dataset from known parameters under the multiple-methods design: an automated method that returns true detections and false positives at every site, and a verified method, free of false positives, at a random subset of sites. The false-positive rate can be homogeneous, vary at random among sites (unstructured), or track a latent gradient correlated with the occupancy covariate (structured confusion, as when a soundalike species becomes commoner along the same gradient as the target).
simulate_fp( nsite = 300, nocc = 20, psi = 0.5, b1 = -0.8, p11 = 0.6, p10 = 0.05, sigma_p10 = 0, p10_conc = NULL, conf_effect = 0, conf_cor = 0, r11 = 0.6, frac_verified = 0.3, nver = 3, a_bio = 0, a_aci = 0, sd_recorder = 0, nrec = 5, frac_confirmed = 0, seed = NULL )simulate_fp( nsite = 300, nocc = 20, psi = 0.5, b1 = -0.8, p11 = 0.6, p10 = 0.05, sigma_p10 = 0, p10_conc = NULL, conf_effect = 0, conf_cor = 0, r11 = 0.6, frac_verified = 0.3, nver = 3, a_bio = 0, a_aci = 0, sd_recorder = 0, nrec = 5, frac_confirmed = 0, seed = NULL )
nsite |
Number of sites. |
nocc |
Number of automated occasions per site. |
psi |
Occupancy probability at the mean of the covariate. |
b1 |
Occupancy slope on the standardized covariate |
p11 |
True-detection probability of the automated method at the mean of the detection covariates. |
p10 |
False-positive probability of the automated method (its base
value; the realized mean over sites is returned as |
sigma_p10 |
Among-site standard deviation of the logit of the false-positive rate (unstructured, logit-normal heterogeneity). 0 for none. |
p10_conc |
Optional concentration of a Beta distribution for the
site-level false-positive rate, with mean |
conf_effect |
Slope of the logit of the false-positive rate on the
confusing gradient |
conf_cor |
Correlation between |
r11 |
Detection probability of the verified method. |
frac_verified |
Probability that a site is verified. |
nver |
Number of verified occasions per verified site. |
a_bio, a_aci
|
Effects of two standardized observation covariates
( |
sd_recorder |
Standard deviation of a recorder random intercept on true detection. 0 by default. |
nrec |
Number of recorders. |
frac_confirmed |
Probability that an automated occasion is checked under the observation-confirmation design of Chambert et al. (2015). 0 (default) for none. See Details. |
seed |
Optional integer seed. |
The occupancy covariate x is standardized, so b1 is the true slope
exactly. The confusing gradient is , with standard normal, so it has unit variance and
correlation with x. In practice w is not observed; it is
returned in sites so that a model given the true structure can serve as a
positive control. With the same seed, the function returns the same
datasets as simulate_fp() of the fpcalib research compendium (v0.1.0,
doi:10.5281/zenodo.23379100), with nocc, frac_verified, nver,
p10_conc, sd_recorder, and nrec corresponding to its K,
verified_coverage, n_verified, p10_beta_conc, sig_u, and nobser
(checked by dev/compare-simulate-fp.R in the package repository).
With frac_confirmed > 0, false observations occur at occupied sites too:
on each occasion, a true observation occurs with probability p11 (and
its covariate effects) at occupied sites, and a false one with the site's
false-positive rate at any site, independently. The occasion is a
detection if either occurs. Checked occasions get the columns n_true
and n_false of auto (0 or 1; NA elsewhere). These draws come after
all others, so the default design reproduces the same random numbers.
A list of class fp_sim with elements data (an fp_data object
whose sites table holds x and w, and whose auto table holds bio,
aci, and recorder) and truth (the generating values, the realized
mean false-positive rate p10_marginal, the realized correlation
between w and x, and the latent states z).
# homogeneous false positives s0 <- simulate_fp(nsite = 100, nocc = 10, seed = 1) s0 # structured confusion s1 <- simulate_fp(nsite = 100, nocc = 10, conf_effect = 1.5, conf_cor = 0.7, seed = 1) s1$truth$p10_marginal# homogeneous false positives s0 <- simulate_fp(nsite = 100, nocc = 10, seed = 1) s0 # structured confusion s1 <- simulate_fp(nsite = 100, nocc = 10, conf_effect = 1.5, conf_cor = 0.7, seed = 1) s1$truth$p10_marginal
Generates new detections from an fp_occu fit, keeping everything else
of the original data: the sites and their covariates, the automated
occasions with their observation covariates and recorders, which sites
and occasions were verified, and which occasions were checked under the
observation-confirmation design. Only the latent occupancy states and the
detections are drawn anew, from the fitted model.
## S3 method for class 'fp_occu' simulate(object, nsim = 1, seed = NULL, truth = c("mean", "posterior"), ...)## S3 method for class 'fp_occu' simulate(object, nsim = 1, seed = NULL, truth = c("mean", "posterior"), ...)
object |
An |
nsim |
Number of datasets. |
seed |
Optional integer seed. |
truth |
Parameter values to simulate from: |
... |
Unused. |
The recorder effects, when the model has them, are kept at their values in
the chosen draw (or their posterior means), so the simulated data have the
same recorders with the same effects. Site effects on the false-positive
rate (fp_occu(random_fp = TRUE)) are kept at their values in the draw
with truth = "posterior"; with truth = "mean", whose site effects
would be shrunk towards zero, new ones are drawn from a normal
distribution with the posterior mean of sd_fp_site as standard
deviation. Under the multiple-methods
design, an automated occasion at an occupied site is a detection with
probability and at an unoccupied site with probability
; verified occasions are detections with probability
at occupied sites only. Under the observation-confirmation
design, true and false observations are drawn independently, as in
simulate_fp().
A list of nsim elements, each a list with data (an fp_data
object with the structure of the original data) and truth (the
generating values on the scale of the output of fp_occu(), the latent
states z, the false-positive rate of each site p10_site, and its mean
p10_bar).
fp_calibrate_fit(), simulate_fp()
## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) new <- simulate(m, nsim = 2, seed = 1) new[[1]]$data new[[1]]$truth$p10_bar ## End(Not run)## Not run: sim <- simulate_fp(nsite = 200, nocc = 10, seed = 1) m <- fp_occu(sim$data, occ = ~ x, fp = ~ 1, random_recorder = FALSE) new <- simulate(m, nsim = 2, seed = 1) new[[1]]$data new[[1]]$truth$p10_bar ## End(Not run)