Conventional longitudinal data analysis methods typically assume that outcomes are independent of the data-collection schedule. However, the independence assumption may be violated when an event triggers outcome assessment in between prescheduled follow-up visits. For example, patients initiating warfarin therapy who experience poor anticoagulation control may have extra physician visits to monitor the impact of necessary dose changes. Observation times may therefore be associated with outcome values, which may introduce bias when estimating the effect of covariates on outcomes using standard longitudinal regression methods. We consider a joint model approach with two components: a semi-parametric regression model for longitudinal outcomes and a recurrent event model for observation times. The semi-parametric model includes a parametric specification for covariate effects, but allows the effect of time to be unspecified. We formulate a framework of outcome-observation dependence mechanisms to describe conditional independence between the outcome and observation-time processes given observed covariates or shared latent variables. We generalize existing methods for continuous outcomes by accommodating any combination of mechanisms through the use of observation-level weights and/or patient-level latent variables. We develop new methods for binary outcomes, while retaining the flexibility of a semi-parametric approach. We extend these methods to account for discontinuous risk intervals in which patients enter and leave the at-risk set multiple times during the study. Our methods are based on counting process approaches, rather than relying on possibly intractable likelihood-based or pseudo-likelihood-based approaches, and provide marginal, population-level inference. In simulations, we evaluate the statistical properties of our proposed methods. Comparisons are made to `naive' approaches that do not account for outcome-dependent observation times. We illustrate the utility of our proposed methods using data from a randomized trial of interventions designed to improve adherence to warfarin therapy and a randomized trial of malaria vaccines among children in Mali.