Main Modules¶
stormtracks.analysis – Analysis of results¶
- class stormtracks.analysis.StormtracksAnalysis(year, is_setup_logging=False)¶
Provides a variety of ways of analysing tracking performance
Makes extensive use of its results_manager to load/save results. Used by the pyro code to farm out jobs across the cluster. To a large extent replaces the manual analysis steps. :param year: year on which to run analysis :param is_setup_logging: whether to setup logging (useful in ipython)
- get_good_matches(ensemble_member, config)¶
Either loads or generates (and saves) good_matches
- get_matching_configs(**kwargs)¶
Allows for easy filtering of current config options
- get_vort_tracks_by_date(ensemble_member, config)¶
Either loads or generates (and saves) vort_tracks_by_date
- good_matches_key(config)¶
Returns the good_matches key for the config options
- list_stats(ensemble_member=0, sort_on='avgdist', active_configs={})¶
Runs through all statistics for the requested ensemble member and compiles stats
sorts on the requested column and only looks at the active_configs. This makes it easy to e.g. compare all scale 1 or all 850 configuration options.
- next_best_track()¶
Moves each plotter’s best track on by one
- plot()¶
Uses each plotter to plot the current scene
- prev_best_track()¶
Moves each plotter’s best track back by one
- print_stats(ensemble_member=0, sort_on='avgdist')¶
Prints the stats
- run_analysis(ensemble_member, force_regen=False)¶
For each set of config options, run a tracking analysis and store the results
- run_ensemble_matches_analysis(num_ensemble_members=56, force_regen=False)¶
Looks at a particular config and runs a matching algortihm against each track
- run_individual_analysis(ensemble_member, config)¶
Runs a given analysis based on config dict
- run_position_analysis(sort_on='avgdist', active_configs={}, force_regen=False, num_ensemble_members=56)¶
Runs a positional analysis on the given sort_on col
If sort_on is e.g. avg_dist, summed av dist for each of the active configs are calc’d and they are ranked in terms of which is lowest
- run_wld_analysis(active_configs={}, num_ensemble_members=56)¶
Runs a win/lose/draw analysis on all ensemble members
If a track from a particular analysis has a lower average dist it is said to have ‘won’ i.e. track for Wilma in pressure_level:850/scale:2/tracker:nearest neighbour has a lower av dist than pl:995/../.. .
- set_year(year)¶
Sets the year, loading best_tracks and setting up results_manager appropriately
- setup_analysis()¶
Sets up the current configuration options
- setup_display(ensemble_member, active_configs={})¶
Sets up plotters for displaying of results
- vort_tracks_by_date_key(config)¶
Returns the vort_tracks_by_date key for the config options
- stormtracks.analysis.analyse_ibtracs_data(plot=True)¶
Adds up a freq. distribution and a yearly total of hurricane-timesteps
- stormtracks.analysis.run_ensemble_analysis(stormtracks_analysis, year, num_ensemble_members)¶
Performs a full enesmble analysis on the given year
Searches through and tries to match all tracks across ensemble members without using any best tracks info.
- stormtracks.analysis.run_tracking_stats_analysis(stormtracks_analysis, year, num_ensemble_members=56)¶
Runs a complete tracking analysis, comparing the performance of each configuration option
Compares performance in a variety of ways, e.g. within pressure level or just scale 1.
stormtracks.c20data – C20 Reanalysis Data¶
- class stormtracks.c20data.C20Data(start_year, fields='all', smoothing=False, upscaling=False, verbose=True, pressure_level=850, scale_factor=2)¶
Class used for accessing data from C20 Reanalysis project.
This acts as a wrapper around netCDF4.Datasets and makes it easy to view data. Typically it exposes the psl, vort, and vort4 fields for one ensemble member. (vort4 is calculated using a 4th order vorticity calculation.) It will load these fields, along with corresponding maxima (vorticity) and minima (pressure) each time a new date is set for an object.
Parameters: - start_year – Year from which to take data
- smoothing – Apply smoothing to data fields
- upscaling – Upscale data using cubic splines
- verbose – Prints lots of output
- pressure_level – Which pressure level (850/995 hPa) to use
- scale_factor – how much to scale data by
- close_datasets()¶
Closes all open datasets
- cvorticity(u, v)¶
Calculates the (2nd order) vorticity by calling into a c function
- cvorticity4(u, v)¶
Calculates the (4th order) vorticity by calling into a c function
Algorithm was taken from Walsh’s code
- first_date(ensemble_member=0)¶
Sets date to the first date of the year (i.e. Jan the 1st)
- fourth_order_vorticity(u, v)¶
Calculates the (4th order) vorticity using python
Algorithm was taken from Walsh’s code
- load_datasets(year)¶
Loads datasets for a given year
Just sets up the NetCDF4 objects, doesn’t actually load any data apart from lons/lats and dates.
- next_date(ensemble_member=0)¶
Moves date on by one timestep (6hr)
- prev_date(ensemble_member=0)¶
Moves date back by one timestep (6hr)
- set_date(date, ensemble_member=0)¶
Sets date and loads all data for that date
Will have no effect if there is no difference in date or ensemble_member.
Parameters: - date – date to load
- ensemble_member – ensemble member to load
Returns: date if successful, otherwise None
- set_year(year)¶
Sets a year and loads the relevant dataset
- vorticity(u, v)¶
Calculates the (2nd order) vorticity using python
- class stormtracks.c20data.GlobalEnsembleMember(c20data, ensemble_member=0)¶
Wrapper around a C20Data object
holds state of which ensemble member is currently being analysed
- set_date(date)¶
Sets the date, does nothing if it is the same as the stored date
- set_year(year)¶
Change the year of the stored C20Data object
stormtracks.ibtracsdata – IBTrACS Data¶
- class stormtracks.ibtracsdata.IbStormtrack(year, name)¶
Holds info about an IBTrACS best track
- class stormtracks.ibtracsdata.IbtracsData(data_dir=None, verbose=True)¶
Class used for accessing IBTrACS data
Wraps the underlying NetCDF4 files and extracts the information required from them.
Parameters: - data_dir – directory where IBTrACS NetCDF4 files are held
- verbose – whether to pring lots of output
- load_ibtracks_year(year, basin='NA')¶
Loads a given year’s worth of data
Parameters: - year – year to load
- basin – which basins to load (‘all’ for all of them)
Returns: list of loaded best tracks
- load_wilma_katrina()¶
Loads only best tracks corresponding to Wilma and Katrina (2005)
stormtracks.tracking – Tracking of Features¶
- class stormtracks.tracking.VortMax(date, pos, vort)¶
Holds key info (date, position, vorticity value) about a vorticity maximum.
To serialize this class (or any that contain objects of this class) you must make sure next_vortmax/prev_vortmax are None.
Parameters: - date – date of vortmax
- pos – position (2 element tuple)
- vort – vorticity value
- add_next(vortmax)¶
Used to make doubly linked list of vortmaxes
- class stormtracks.tracking.VortMaxTrack(start_vortmax, ensemble_member)¶
Stores a collection of VortMax objects in a list and adds them to a dict that is accessible through a date for easy access.
- class stormtracks.tracking.VortmaxFinder(gem)¶
Finds all vortmaxes for a given ensemble member
Parameters: gem – GlobalEnsembleMember to use - find_vort_maxima(start_date, end_date, use_upscaled=False)¶
Runs over the date range looking for all vorticity maxima
- class stormtracks.tracking.VortmaxKalmanFilterTracker(ensemble_member, Q_mult=0.001, R_mult=0.1)¶
Uses a Kalman Filter to try to track vorticity maxima
The main idea is to use the innovation parameter from the Kalman Filter estimation process as a measure of how likely it is that a vorticity maxima in a subsequent timestep belongs to the same track as a vorticity maxima in the current timestep
- track_vort_maxima(vortmax_time_series)¶
Uses a generated list of vortmaxes to track them from one timestep to another
Parameters: vortmax_time_series – dict of vortmaxes Returns: self.vort_tracks_by_date (OrderedDict)
- class stormtracks.tracking.VortmaxNearestNeighbourTracker(ensemble_member)¶
Simple nearest neighbour tracker
Assumes that the two nearest points from one timestep to another belong to the same track.
- track_vort_maxima(vortmax_time_series)¶
Uses a generated list of vortmaxes to track them from one timestep to another
Parameters: vortmax_time_series – dict of vortmaxes Returns: self.vort_tracks_by_date (OrderedDict)
stormtracks.tracking – Matching of Tracks¶
- class stormtracks.matching.BestTrackMatch(best_track, store_all_tracks=False)¶
Represents one match between many vorticity tracks
- add_match(match)¶
Checks to see whether the macth should be added, then adds it
Compares the overlap and cumalitive distance between the match to the current average track, and if they are below a limit adds the match, which involves recalculating the average track for this match, and updating some fields.
Parameters: match – match to be added Returns: True if it was added, otherwise False
- av_dist()¶
Returns the average distance between all vorticity tracks
- class stormtracks.matching.CycloneMatch(best_track, cyclone)¶
Represents one match between a best track and a vorticity track
- av_dist()¶
Returns the average distance between the best and vorticity tracks
- class stormtracks.matching.EnsembleMatch(vort_track, store_all_tracks=False)¶
Represents one match between many vorticity tracks
- add_track(vort_track)¶
Checks to see whether the track should be added, then adds it
Compares the overlap and cumalitive distance between the track to the current average track, and if they are below a limit adds the track, which involves recalculating the average track for this match, and updating some fields.
Parameters: vort_track – vorticitiy track to be added Returns: True if it was added, otherwise False
- av_dist()¶
Returns the average distance between all vorticity tracks
- class stormtracks.matching.Match(best_track, vort_track)¶
Represents one match between a best track and a vorticity track
- av_dist()¶
Returns the average distance between the best and vorticity tracks
- stormtracks.matching.combined_match(best_tracks, all_matches)¶
Uses all best tracks and matches to combine all matches for each best track
Parameters: - best_tracks – list of best tracks
- all_matches – all matches to search through
Returns: dict of combined_matches (key: best track, value: list of matches)
- stormtracks.matching.good_matches(matches, dist_cutoff=None, overlap_cutoff=6)¶
Returns all matches that meet the requirements
Requirements are that match.av_dist() < dist_cutoff and match.overlap >= overlap_cutoff
- stormtracks.matching.match_ensemble_vort_tracks_by_date(vort_tracks_by_date_list)¶
Takes the normal output from a tracker and flattens them to a list
- stormtracks.matching.match_vort_tracks_by_date_to_best_tracks(vort_tracks_by_date, best_tracks)¶
Takes all vorticity tracks and best tracks and matches them up
Parameters: - vort_tracks_by_date – dict with dates as keys and lists of vort tracks as values
- best_tracks – list of best tracks
Returns: array of all matches
stormtracks.results – Store and Access Results¶
- exception stormtracks.results.ResultNotFound¶
Simple exception thrown if result cannot be found in results manager or on disk
- class stormtracks.results.StormtracksNumpyResultsManager(name)¶
Super simple key/value store for numpy arrays.
- class stormtracks.results.StormtracksResultsManager(name, cache_loaded=False)¶
Manager class that is responsible for loading and saving all python results
Simple key/value store. Load/saves to settings.OUTPUT_DIR. Saves each result to e.g.:
settings.OUTPUT_DIR/<self.name>/<year>/<ensemble_member>-<key>.pkl
(using that as a directory/filename structure).
- add_result(year, ensemble_member, result_key, result)¶
Adds a given result based on year, ensemble_member and a user chosen result_key
- compress_year(year, delete=False)¶
Compresses a given year’s dir and then deletes that year
- decompress_year(year)¶
Decompresses a given year’s tarball
- delete(year, ensemble_member, result_key)¶
Deletes a specific result from disk
- delete_year(year)¶
Deletes a year (use with caution!)
- get_result(year, ensemble_member, result_key)¶
Returns a saved/already loaded result if it exists, or result from disk
throws a ResultNotFound error if it can’t find the requested result on disk.
- list_ensemble_members(year)¶
List all results saved for a particular year
- list_results(year, ensemble_member)¶
List all results saved for a particular year/ensemble_member
- list_years()¶
List all saved years
- save()¶
Saves all unsaved results that have been added so far