Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OPSIM-1086: remove obsolete code and simplify upkeep of example_scheduler #372

Merged
merged 16 commits into from
Oct 17, 2023
Merged

OPSIM-1086: remove obsolete code and simplify upkeep of example_scheduler #372

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e2d21e4
Move LMC/SMC ra/dec into init, inherit and add updates to defaults fr…
rhiannonlynne Oct 14, 2023
ec2703f
Add basic skyarea unit tests
rhiannonlynne Oct 14, 2023
e479759
Add nside skymap test
rhiannonlynne Oct 14, 2023
67776d8
Move functions dealing with static sky from footprints to skyarea
rhiannonlynne Oct 14, 2023
a0999e2
Remove obsolete functions from footprints
rhiannonlynne Oct 14, 2023
5507a97
Update tests to support removal of standard_goals
rhiannonlynne Oct 14, 2023
38464cf
docstrings/ruff
rhiannonlynne Oct 14, 2023
1be7d55
docstrings/ruff
rhiannonlynne Oct 14, 2023
7c8b215
This is a rearrangement and update of the example_scheduler + "main" …
rhiannonlynne Oct 14, 2023
47fd31b
Support array of target maps returned by SkyAreaGenerator in calc_nor…
rhiannonlynne Oct 14, 2023
6eeb7f8
Make sure example_scheduler can load at any nside
rhiannonlynne Oct 14, 2023
f6860f6
correct path for dust_maps in scheduler
rhiannonlynne Oct 14, 2023
8d4e099
make test of constant weather skip unnecessarily loading the sky
rhiannonlynne Oct 14, 2023
5f3d641
Fix typos
rhiannonlynne Oct 14, 2023
8c08241
Isort
rhiannonlynne Oct 14, 2023
65961d4
Resolve test errors in absence of scheduler dustmap data
rhiannonlynne Oct 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
474 changes: 198 additions & 276 deletions rubin_sim/scheduler/example/example_scheduler.py
View file
Open in desktop

Large diffs are not rendered by default.

18 changes: 6 additions & 12 deletions rubin_sim/scheduler/model_observatory/model_observatory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,12 @@
from rubin_sim.scheduler.features import Conditions
from rubin_sim.scheduler.model_observatory import KinemModel
from rubin_sim.scheduler.utils import create_season_offset, set_default_nside
from rubin_sim.site_models import (
Almanac,
CloudData,
ScheduledDowntimeData,
SeeingData,
SeeingModel,
UnscheduledDowntimeData,
)

# For backwards compatibility
from rubin_sim.site_models import Almanac, CloudData
from rubin_sim.site_models import ConstantCloudData as NoClouds
from rubin_sim.site_models import ConstantSeeingData as NominalSeeing
from rubin_sim.site_models import ScheduledDowntimeData, SeeingData, SeeingModel, UnscheduledDowntimeData
from rubin_sim.utils import (
Site,
_angular_separation,
Expand All @@ -30,10 +28,6 @@
survey_start_mjd,
)

# For backwards compatibility
from rubin_sim.site_models import ConstantCloudData as NoClouds
from rubin_sim.site_models import ConstantSeeingData as NominalSeeing


class ModelObservatory:
"""A class to generate a realistic telemetry stream for the scheduler"""
Expand Down
155 changes: 79 additions & 76 deletions rubin_sim/scheduler/schedulers/core_scheduler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
__all__ = ("CoreScheduler",)

import logging
import reprlib
import time
from collections import OrderedDict
from copy import deepcopy
Expand All @@ -17,22 +16,29 @@


class CoreScheduler:
"""Core scheduler that takes completed observations and observatory status and requests observations
"""Core scheduler that takes requests observations, reports observatory
status and provides completed observations.

Parameters
----------
surveys : list (or list of lists) of rubin_sim.scheduler.survey objects
A list of surveys to consider. If multiple surveys return the same highest
reward value, the survey at the earliest position in the list will be selected.
Can also be a list of lists to make heirarchical priorities.
nside : int
A list of surveys to consider.
If multiple surveys return the same highest reward value, the survey
at the earliest position in the list will be selected.
Can also be a list of lists to make heirarchical priorities ('tiers').
nside : `int`
A HEALpix nside value.
camera : str ('LSST')
Which camera to use for computing overlapping HEALpixels for an observation.
Can be 'LSST' or 'comcam'
conditions : a rubin_sim.scheduler.features.Conditions object (None)
An object that hold the current conditions and derived values (e.g., 5-sigma depth). Will
generate a default if set to None.
camera : `str`
Which camera to use to compute the correspondence between
visits and HEALpixels when recording observations.
Can be 'LSST' or 'comcam'.
rotator_limits : `list` [`float`, `float`]
Limits for the camera rotator.
log : `logging.Logger`
If None, a new logger is created.
keep_rewards : `bool`
Flag as to whether to record the rewards and basis function values
(this can be useful for schedview).
"""

def __init__(
Expand All @@ -44,22 +50,6 @@ def __init__(
log=None,
keep_rewards=False,
):
"""
Parameters
----------
surveys : list (or list of lists) of rubin_sim.scheduler.survey objects
A list of surveys to consider. If multiple surveys return the same highest
reward value, the survey at the earliest position in the list will be selected.
Can also be a list of lists to make heirarchical priorities.
nside : int
A HEALpix nside value.
camera : str ('LSST')
Which camera to use for computing overlapping HEALpixels for an observation.
Can be 'LSST' or 'comcam'
rotator_limits : sequence of floats
keep_rewards : `bool`
Keep records of rewards and basis funciton values
"""
self.keep_rewards = keep_rewards
# Use integer ns just to be sure there are no rounding issues.
self.mjd_perf_counter_offset = np.int64(Time.now().mjd * 86400000000000) - time.perf_counter_ns()
Expand All @@ -74,7 +64,8 @@ def __init__(

# initialize a queue of observations to request
self.queue = []
# The indices of self.survey_lists that provided the last addition(s) to the queue
# The indices of self.survey_lists that provided
# the last addition(s) to the queue
self.survey_index = [None, None]

# If we have a list of survey objects, convert to list-of-lists
Expand All @@ -85,7 +76,8 @@ def __init__(
self.nside = nside
hpid = np.arange(hp.nside2npix(nside))
self.ra_grid_rad, self.dec_grid_rad = _hpid2_ra_dec(nside, hpid)
# Should just make camera a class that takes a pointing and returns healpix indices
# Should just make camera a class that takes a pointing
# and returns healpix indices
if camera == "LSST":
self.pointing2hpindx = HpInLsstFov(nside=nside)
elif camera == "comcam":
Expand All @@ -106,7 +98,8 @@ def add_observations_array(self, obs):
"""Like add_observation, but for passing many observations at once.

Assigns overlapping HEALpix IDs to each observation, then passes
the observation array and constructed observations + healpix id to each survey.
the observation array and constructed observations + healpix id
to each survey.
"""
obs.sort(order="mjd")
# Generate list-of-lists for HEALPix IDs for each pointing
Expand Down Expand Up @@ -142,7 +135,8 @@ def add_observation(self, observation):
----------
observation : dict-like
An object that contains the relevant information about a
completed observation (e.g., mjd, ra, dec, filter, rotation angle, etc)
completed observation
(e.g., mjd, ra, dec, filter, rotation angle, etc)
"""

# Find the healpixel centers that are included in an observation
Expand All @@ -156,17 +150,20 @@ def update_conditions(self, conditions_in):
Parameters
----------
conditions : dict-like
The current conditions of the telescope (pointing position, loaded filters, cloud-mask, etc)
The current conditions of the telescope (pointing position,
loaded filters, cloud-mask, etc)
"""
# Add the current queue and scheduled queue to the conditions
self.conditions = conditions_in
# put the local queue in the conditions
self.conditions.queue = self.queue

# Check if any surveys have upcomming scheduled observations. Note that we are accumulating
# all of the possible scheduled observations, so it's up to the user to make sure things don't
# collide. The ideal implementation would be to have all the scheduled observations in a
# single survey objects, presumably at the highest tier of priority.
# Check if any surveys have upcomming scheduled observations.
# Note that we are accumulating all of the possible scheduled
# observations, so it's up to the user to make sure things don't
# collide. The ideal implementation would be to have all the
# scheduled observations in a single survey objects,
# presumably at the highest tier of priority.

all_scheduled = []
for sl in self.survey_lists:
Expand All @@ -184,9 +181,10 @@ def update_conditions(self, conditions_in):

def _check_queue_mjd_only(self, mjd):
"""
Check if there are things in the queue that can be executed using only MJD and not full conditions.
This is primarly used by sim_runner to reduce calls calculating updated conditions when they are not
needed.
Check if there are things in the queue that can be executed
using only MJD and not full conditions.
This is primarily used by sim_runner to reduce calls calculating
updated conditions when they are not needed.
"""
result = False
if len(self.queue) > 0:
Expand All @@ -198,18 +196,19 @@ def _check_queue_mjd_only(self, mjd):

def request_observation(self, mjd=None):
"""
Ask the scheduler what it wants to observe next

Parameters
----------
mjd : float (None)
The Modified Julian Date. If None, it uses the MJD from the conditions from the
last conditions update.

Returns
-------
observation object (ra,dec,filter,rotangle)
Returns None if the queue fails to fill
Ask the scheduler what it wants to observe next

Parameters
----------
mjd : float (None)
The Modified Julian Date.
If None, it uses the MJD from the conditions from the
last conditions update.

Returns
-------
observations : observation object (ra,dec,filter,rotangle)
Returns None if the queue fails to fill
"""
if mjd is None:
mjd = self.conditions.mjd
Expand All @@ -219,7 +218,8 @@ def request_observation(self, mjd=None):
if len(self.queue) == 0:
return None
else:
# If the queue has gone stale, flush and refill. Zero means no flush_by was set.
# If the queue has gone stale, flush and refill.
# Zero means no flush_by was set.
if (IntRounded(mjd) > IntRounded(self.queue[0]["flush_by_mjd"])) & (
self.queue[0]["flush_by_mjd"] != 0
):
Expand Down Expand Up @@ -250,8 +250,8 @@ def request_observation(self, mjd=None):

def _fill_queue(self):
"""
Compute reward function for each survey and fill the observing queue with the
observations from the highest reward survey.
Compute reward function for each survey and fill the observing queue
with the observations from the highest reward survey.
"""

# If we loaded the scheduler from a pickle, self.keep_rewards
Expand Down Expand Up @@ -286,8 +286,8 @@ def _fill_queue(self):
else:
to_fix = np.where(np.isnan(rewards) == True)
rewards[to_fix] = -np.inf
# Take a min here, so the surveys will be executed in the order they are
# entered if there is a tie.
# Take a min here, so the surveys will be executed in the order
# they are entered if there is a tie.
self.survey_index[1] = np.min(np.where(rewards == np.nanmax(rewards)))
# Survey return list of observations
result = self.survey_lists[self.survey_index[0]][self.survey_index[1]].generate_observations(
Expand All @@ -300,23 +300,26 @@ def _fill_queue(self):
self.log.warning(f"Failed to fill queue at time {self.conditions.mjd}")

def get_basis_functions(self, survey_index=None, conditions=None):
"""Get the basis functions for a specific survey, in provided conditions.
"""Get the basis functions for a specific survey,
in provided conditions.

Parameters
----------
survey_index : `List` [`int`], optional
A list with two elements: the survey list and the element within that
survey list for which the basis function should be retrieved. If ``None``,
use the latest survey to make an addition to the queue.
conditions : `rubin_sim.scheduler.features.conditions.Conditions`, optional
The conditions for which to return the basis functions. If ``None``, use
the conditions associated with this sceduler. By default None.
survey_index : `List` [`int`]
A list with two elements: the survey list and the element
within that survey list for which the basis function should be
retrieved. If ``None``, use the latest survey to make an addition
to the queue.
conditions : `rubin_sim.scheduler.features.conditions.Conditions`
The conditions for which to return the basis functions.
If ``None``, use the conditions associated with this scheduler.

Returns
-------
basis_funcs : `OrderedDict` ['str`, `rubin_sim.scheduler.basis_functions.basis_functions.Base_basis_function`]
A dictionary of the basis functions, where the keys are names for the basis functions and the values
are the functions themselves.
basis_funcs : `OrderedDict` ['str`,
`rubin_sim.scheduler.basis_functions.basis_functions.Base_basis_function`]
A dictionary of the basis functions, where the keys are names for
the basis functions and the values are the functions themselves.
"""

if survey_index is None:
Expand All @@ -340,19 +343,19 @@ def get_healpix_maps(self, survey_index=None, conditions=None):

Parameters
----------
survey_index : `List` [`int`], optional
A list with two elements: the survey list and the element within that
survey list for which the maps that should be retrieved. If ``None``,
use the latest survey to make an addition to the queue.
conditions : `rubin_sim.scheduler.features.conditions.Conditions`, optional
survey_index : `List` [`int`], opt
A list with two elements: the survey list and the element within
that survey list for which the maps that should be retrieved.
If ``None``, use the latest survey which added to the queue.
conditions : `rubin_sim.scheduler.features.conditions.Conditions`, opt
The conditions for the maps to be returned. If ``None``, use
the conditions associated with this sceduler. By default None.

Returns
-------
basis_funcs : `OrderedDict` ['str`, `numpy.ndarray`]
A dictionary of the maps, where the keys are names for the maps and
values are the numpy arrays as used by ``healpy``.
A dictionary of the maps, where the keys are names for the maps
and values are the numpy arrays as used by ``healpy``.
"""

if survey_index is None:
Expand Down Expand Up @@ -467,7 +470,7 @@ def _repr_markdown_(self):
return str(self)

def surveys_df(self, tier):
"""Create a pandas.DataFrame describing rewards from surveys in one list.
"""Create a pandas.DataFrame describing rewards from surveys.

Parameters
----------
Expand Down
Loading
Loading