o f 2019-08-14T21:11:08.486000000 /Date(1565568000000+0000) -> 2019-08-12T00:00:00.000000000 NrcSs(g|]}t|dd|vr dndqS)+i)int.0_r!r!r" Ns(z _parse_dates..ms)unit)rrrr _parse_dates to_datetimevalues)datesresr!r!r"r.Ds  r.c@seZdZdZdS)DatastreamExceptionz Exception class for Datastream N)__name__ __module__ __qualname____doc__r!r!r!r"r3Rsr3cs(djttfdd}|S)z& Lazy-evaluated property of an object __lazy__cs$t|s t||t|S)N)hasattrsetattrgetattrself attr_namefnr!r"_lazy_property[s  z%lazy_property.._lazy_property)r4propertyr)r@rAr!r>r" lazy_propertyWs rCc@s*eZdZdZd=ddZeddZdd Zd>d d Ze d d Z e ddZ ddZ ddZ e   d?ddZeddZddZd@ddZdAddZ   dBd d!Zd>d"d#Zd>d$d%Zd>d&d'Zd@d(d)Zd*d+Zd,d-Zd.d/ZdCd0d1ZdDd3d4Zd@d5d6ZdEd7d8Zed9d:Z d>d;d<Z!dS)F Datastreamzb Python interface to the Refinitiv Datastream API via Datastream Web Services (DSWS). TNcKsr||_d|_d|_d|_t|tr||d|_n |dur d|_ntd|dt |_ ||_ ||_ | ||dS)aEstablish a connection to the Python interface to the Refinitiv Datastream (former Thomson Reuters Datastream) API via Datastream Web Services (DSWS). username / password - credentials for the DSWS account. raise_on_error - If True then error request will raise a "DatastreamException", otherwise either empty dataframe or partially retrieved data will be returned proxy - URL for the proxy server. Valid values: (a) None: no proxy is used (b) string of format "host:port" or "username:password@host:port" Note: credentials will be saved in memory. In case if this is not desirable for security reasons, call the constructor having None instead of values and manually call renew_token(username, password) when needed. A custom REST API url (if necessary for some reasons) could be provided via "url" parameter. N)httphttpsz/Proxy parameter should be either None or stringurl)raise_on_error last_request last_metadata_last_response_rawrr_proxy ValueErrorpop_URL_url _username _password renew_token)r=usernamepasswordrHproxykwargsr!r!r"__init__ks zDatastream.__init__cCs ttdS)z Some useful links N)print_INFOr!r!r!r"infos zDatastream.infoc Cs|j|}||dd|_d|_ztj|||jd}|j|jd<Wnty4}zt||jd<d}~wwzt |jd}|jd<Wnt j yV}zt d|d}~wwd|vr|d}|ddurm|d |d7}|d |d }||jd<t ||jdS) z% Call to the POST method of DSWS API N)rGrequesterror)jsonproxiesresponser]z#Server response could not be parsedCodeSubCode/z: Message) rPrIrJrequestspostrLtext Exceptionrr^loadsJSONDecodeErrorr3) r=methodr\rGr2er`codeerrormsgr!r!r" _api_posts2     zDatastream._api_postcCs|dus|durtddS||d}t|d||_t|jdd|jd<t|jdt dtj t d|jd <dS) z# Request new token from the server Nz;sz*Datastream._parse_meta..Currencyrlevel) rr set_indexreplaceindexnameconcatxsT)rr2rnamescurr!rr" _parse_meta-s   zDatastream._parse_metac sLd}td}fddD}i}i|D]u}|d}i|<i||<|dD]Pd}dd krS|jrItd d d |d |tj|d <nddkrdt||d <n||d <fddD||d <q+|durd g}tj||d|<qtjddj ||d<| |fS)zE Parse one response (either 'DataResponse' or one of 'DataResponses')DataTypeValuesDatescsi|] }|dvr||qS))rrr!r()r2r!r"rFz)Datastream._parse_one..DataType SymbolValuesrTyper"Symbolz"("z"): csi|] }|dkr||qS)rr!r()vr!r"rZrN)rrrr) r.rHr3mathnanrrrunstackr sort_indexr) r=r2rr1res_metard data_typevaluer!)r2rr" _parse_oneBs2     zDatastream._parse_onecs|d|vr|d\}}|_|r||fS|Sd|vr:fdd|dD}dd|D_|r3|Sdd|DStd)a Parse raw JSON response If return_metadata is True, then result is tuple (dataframe, metadata), where metadata is a dictionary. Otherwise only dataframe is returned. In case of response being constructed from several requests (method request_many()), then the result is a list of parsed responses. Here again, if return_metadata is True then each element is a tuple (dataframe, metadata), otherwise each element is a dataframe. DataResponse DataResponsescsg|]}|qSr!)r)r)rr<r!r"r+usz-Datastream.parse_response..cSg|]}|dqSrr!r(r!r!r"r+vcSr)rr!r(r!r!r"r+wrz0Neither DataResponse nor DataResponses are found)rrJr3)r=r`return_metadatar2rresultsr!r<r"parse_responsees zDatastream.parse_responsercszdurtdtjfddt|Dddd}|}t|}|dj d|_ d|j _ |S)a6 Request usage statistics date - if None, stats for the current month will be fetched, otherwise for the month which contains the specified date. months - number of consecutive months prior to "date" for which stats should be retrieved. Nnowc s*g|]}jddtj|ddqS)STATSz DS.USERSTATSTr)rroffsets MonthBegin)r)nr r=r!r"r+s  z/Datastream.usage_statistics..z Start Datez%B %Y) rr normalizerrrangerrrdtrrr)r=r monthsrr2r!rr"usage_statistics{s  zDatastream.usage_statisticsc  Cs|j||||||||d} || } | |_|j| dd\} } |r(| jddd} nt| jjddkr;| s;| jddd} |rA| | fS| S)aZ Fetch the data from Datastream for a set of tickers and parse results. tickers - ticker or symbol, or list of symbols fields - field or list of fields date_from, date_to - date range (used only if "date" is not specified) freq - frequency of data: daily('D'), weekly('W') or monthly('M') static - True for static (snapshot) requests IsExpression - if True, it will explicitly assume that list of tickers contain expressions. Otherwise it will try to infer it. return_metadata - if True, then tuple (data, metadata) will be returned, otherwise only data always_multiindex - if True, then even for 1 ticker requested, resulting dataframe will have multiindex (ticker, date). If False (default), then request of single ticker will result in a dataframe indexed by date only. Notes: - several fields should be passed as a list, and not as a comma-separated string! - if no fields are provided, then the default field will be fetched. In this case the column might not have any name in the resulting dataframe. Result format depends on the number of requested tickers and fields: - 1 ticker - DataFrame with fields in column names (in order to keep multiindex even for single ticker, set `always_multiindex` to True) - several tickers - DataFrame with fields in column names and MultiIndex (ticker, date) - static request - DataFrame indexed by tickers and with fields in column names Some of available fields: P - adjusted closing price PO - opening price PH - high price PL - low price VO - volume, which is expressed in 1000's of shares. UP - unadjusted price OI - open interest MV - market value EPS - earnings per share DI - dividend index MTVB - market to book value PTVB - price to book value ... )rrrrT)rrrdropr)rr\rKr reset_indexlenrlevels)r=tickersrrrrrrralways_multiindexrrawrrr!r!r"fetchs 2 zDatastream.fetchcC|j|gd||dddS)zGet Open, High, Low, Close prices and daily Volume for a given ticker. ticker - ticker or symbol date_from, date_to - date range (used only if "date" is not specified) )POPHPLPVODFrrrr=rrrr!r!r" get_OHLCVzDatastream.get_OHLCVcCr)zGet Open, High, Low and Close prices for a given ticker. ticker - ticker or symbol date_from, date_to - date range (used only if "date" is not specified) )rrrrrFrrrr!r!r"get_OHLCrzDatastream.get_OHLCcCs|j|d||dddS)zGet Close price for a given ticker. ticker - ticker or symbol date_from, date_to - date range (used only if "date" is not specified) rrFrrrr!r!r" get_prices zDatastream.get_pricecCs&|rddg}nt}|jd||ddS)a Get a list of all constituents of a given index. index_ticker - Datastream ticker for index only_list - request only list of symbols. By default the method retrieves many extra fields with information (various mnemonics and codes). This might pose some problems for large indices like Russel-3000. If only_list=True, then only the list of symbols and names are retrieved. NOTE: In contrast to retired DWE interface, DSWS does not support fetching historical lists of constituents. MNEMNAMELTr) _FLDS_XREFr)r= index_ticker only_listrr!r!r"get_constituentss zDatastream.get_constituentscs|j|ddd}tdd|jD}i}tddD]fdd |D}||j|d |<qt|d  }||d kj dd }|S)zC Get all listings and their symbols for the given security QTEALLTrcSsh|]}|ddqS)Nr!r(r!r!r" sz.Datastream.get_all_listings..rcsi|] }|d|qS)02dr!)r)cindr!r"rz/Datastream.get_all_listings..columnsrr)axis) rrrrkeysrenamerr swaplevelrall)r=rr2rdfcolsr!rr"get_all_listingsszDatastream.get_all_listingscCs|j|tddS)z8 Get codes and symbols for the given securities Tr)rr)r=rr!r!r" get_codesszDatastream.get_codescCsl|j|dddd}ttjdd}|j|dd}t|ttjfr4z|j|}W|St y3Y|Sw|S)z[ Get asset types for a given list of symbols Note: the method does not TYPETF)rr AssetTypeNamer)on) rrSeries_ASSET_TYPE_CODESto_framerrrlocKeyError)r=symbolsr2rr!r!r"get_asset_typess  zDatastream.get_asset_typescCsr|rddg}nt}|jd|d|dd}d|d<|r2|jd|d|dd}d |d<t||g}||jd kS) a Get list of all contracts for a given futures market market_code - Datastream mnemonic for a market (e.g. LLC for the Brent Crude Oil, whose contracts have mnemonics like LLC0118 for January 2018) only_list - request only list of symbols. By default the method retrieves many extra fields with information (currency, lot size, last trading date, etc). If only_list=True, then only the list of symbols and names are retrieved. include_dead - if True, all delisted/expired contracts will be fetched as well. Otherwise only active contracts will be returned. rrLFUTrTractiverFr)_FLDS_XREF_FUTrrrr)r= market_coder  include_deadrr2res2r!r!r"get_futures_contracts*s z Datastream.get_futures_contracts 1951-01-01c Cs|j|d||d}|jd}|j|gd||djdd}i}|jD]}z |j|d||d} Wn ty<Yq$w| ||<q$t|jS)a Construct the vintage matrix for a given economic series. Requires subscription to Thomson Reuters Economic Point-in-Time (EPiT). Vintage matrix represents a DataFrame where columns correspond to a particular period (quarter or month) for the reported statistic and index represents timestamps at which these values were released by the respective official agency. I.e. every line corresponds to all available reported values by the given date. For example: >> DS.get_epit_vintage_matrix('USGDP...D', date_from='2015-01-01') 2015-02-15 2015-05-15 2015-08-15 2015-11-15 2015-04-29 16304.80 NaN NaN NaN 2015-05-29 16264.10 NaN NaN NaN 2015-06-24 16287.70 NaN NaN NaN 2015-07-30 16177.30 16270.400 NaN NaN 2015-08-27 16177.30 16324.300 NaN NaN 2015-09-25 16177.30 16333.600 NaN NaN 2015-10-29 16177.30 16333.600 16394.200 NaN 2015-11-24 16177.30 16333.600 16417.800 NaN From the matrix it is seen for example, that the advance GDP estimate for 2015-Q1 (corresponding to 2015-02-15) was released on 2015-04-29 and was equal to 16304.80 (B USD). The first revision (16264.10) has happened on 2015-05-29 and the second (16287.70) - on 2015-06-24. On 2015-07-30 the advance GDP figure for 2015-Q2 was released (16270.400) together with update on the 2015-Q1 value (16177.30) and so on. REL1rrr)RELD1RELD2RELD3r)howRELV)rdropnarr3rrr8r) r=mnemonicrrrel1date_0reld123r2r _tmpr!r!r"get_epit_vintage_matrixDs !    z"Datastream.get_epit_vintage_matrixcsp|r |j|d|ddn |j|d|ddjdfddtd|r%d nd D}tj|jd d }|S) a& Return initial estimate and first revisions of a given economic time series and a given period. Requires subscription to Thomson Reuters Economic Point-in-Time (EPiT). "Period" parameter should represent a date which falls within a time period of interest, e.g. 2016 Q4 could be requested with the period='2016-11-15' for example. By default up to 20 values is returned unless argument "relh50" is set to True (in which case up to 50 values is returned). RELH50T)rrRELHrcs8i|]}jd|dkrjd|jd|qS)z RELHD%02drz RELHV%02d)r&r)irr!r"rs z1Datastream.get_epit_revisions..r3rzRELHP r!)rilocrrr#r&r)r=r:periodrelh50r2r!rDr"get_epit_revisionsws   zDatastream.get_epit_revisionsc s|dkrtd|dkrtdfddt|D}|}g}|D]8}|jddd}d |j_d d|jDd t d d |d<|j ddd}fdd|jD|_| |q(t |}dD] }t j||dd||<qj|S)a Return the next date of release (NDoR) for a given economic series. Could return results for up to 12 releases in advance. Returned fields: DATE - Date of release (or start of range where the exact date is not available) DATE_LATEST - End of range when a range is given TIME_GMT - Expected time of release (for "official" dates only) DATE_FLAG - Indicates whether the dates are "official" ones from the source, or estimated by Thomson Reuters where "officials" not available REF_PERIOD - Corresponding reference period for the release TYPE - Indicates whether the release is for a new reference period ("NewValue") or an update to a period for which there has already been a release ("ValueUpdate") z2Only up to 12 months in advance could be requestedrz-n_releases smaller than 1 does not make sensecs&g|]}jd|dddqS)DS.NDORrTr)rrB) mnemonicsr=r!r"r+sz5Datastream.get_next_release_dates..TrMnemoniccSsg|] }|ddqS)r*r)splitr(r!r!r"r+srrKr ReleaseNo)rcsg|] }|ddqS)r*r)rr()ndor_idxr!r"r+r)DATE DATE_LATEST REF_PERIODcoerce)errors)rhrrrrrrrr'rrrrrrr/) r=rL n_releasesreqs res_parsedr2rxcolr!)rLrPr=r"get_next_release_datess*  z!Datastream.get_next_release_datescCs*|jdgddd}||ddkdS)z7 List of mnemonics for holidays in different countries HOLIDAYS)rENAMEGEOGNGEOGCTrrrr_)r sort_values)r=r2r!r!r"vacations_listszDatastream.vacations_listcCst|tr|g}|j}||j|}t||j}|r'tdd||j |j ||dd}t |dkrG|j dddfj |ddS|j dddfjddj|d d d |S) a Get list of trading dates for a given countries (speficied by ISO-2c) Returning dataframe will contain values 1 and NaN, where 1 identifies the business day. So by multiplying this list with the price time series it will remove padded values on non-trading days. Example: DS.get_trading_days(['US', 'UK', 'RS'], date_from='2010-01-01') zUnknowns ISO codes: z, r3rNrr!rrr_r)rrrar_isinset differencer3rrrrrFr%rrr)r= countriesrrvacsmnems missing_isosr2r!r!r"get_trading_dayss  zDatastream.get_trading_days)TN)NN)NNNNFNT)F)Nr)NNNNFNFF)FF)r1Nr)"r4r5r6r7rX staticmethodr[rorSrBrrr\rrrrrrrrrrr rrr)r0r?rIr[rCrarir!r!r!r"rDgsR (      H  #  G    3 - rD)r7rxr^r functoolsrrepandasrrOrNrr,r$rZr#r.rhr3rCrDr!r!r!r"sF