ó ÍÿŠTc@slddlZddlZddlZddlZddlZddlZddlZddlmZm Z ddl m Z ddl m Z mZddlmZddlmZddlmZddlmZmZmZdd lmZmZd Zd Zd efd „ƒYZdefd„ƒYZdefd„ƒYZ defd„ƒYZ!d„Z"d„Z#dS(iÿÿÿÿN(tDeferredtsucceed(tsequence_to_ranges(tgather_resultst spawn_process(t fetch_async(t touch_file(tbpickle(tPackageTaskHandlerConfigurationtPackageTaskHandlertrun_task_handler(tUnknownHashIDRequesttFakePackageStorei iôtPackageReporterConfigurationcBseZdZd„ZRS(s=Specialized configuration for the Landscape package-reporter.cCs8tt|ƒjƒ}|jddtddddƒ|S(sl Specialize L{Configuration.make_parser}, adding options reporter-specific options. s--force-apt-updatetdefaulttactiont store_truethelpsForce running apt-update.(tsuperR t make_parsert add_optiontFalse(tselftparser((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRs (t__name__t __module__t__doc__R(((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR stPackageReportercBsÝeZdZeZdZdZdZdZe Z d„Z d„Z d„Z d„Zd „Zd „Zd „Zd „Zd „Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„ZRS(stReport information about the system packages. @cvar queue_name: Name of the task queue to pick tasks from. treporters/usr/lib/landscape/apt-updates/etc/apt/sources.lists/etc/apt/sources.list.dcsÓtˆ_tƒ}|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|jdƒ|S( Ncs ˆjƒS(N(tget_session_id(tx(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyt<scs ˆjƒS(N(trun_apt_update(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR>scs ˆjƒS(N(tfetch_hash_id_db(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRAscs ˆjƒS(N(tuse_hash_id_db(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRDscs ˆjƒS(N(t handle_tasks(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRGscs ˆjƒS(N(tremove_expired_hash_id_requests(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRJscs ˆjƒS(N(trequest_unknown_hashes(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRMscs ˆjƒS(N(tdetect_changes(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRPs(Rt _got_taskRt addCallbacktcallbacktNone(Rtresult((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytrun7s   cCs|jj||jtƒS(N(t_brokert send_messaget _session_idtTrue(Rtmessage((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR.Us cs,‡fd†}ˆjƒ}|j|ƒ|S(sî Fetch the appropriate pre-canned database of hash=>id mappings from the server. If the database is already present, it won't be downloaded twice. The format of the database filename is __, and it will be downloaded from the HTTP directory set in config.package_hash_id_url, or config.url/hash-id-databases if the former is not set. Fetch failures are handled gracefully and logged as appropriate. cs¼ˆdkrdStjjˆƒr&dSˆjƒ}|sItjdƒdSt|tjjˆƒƒ‰‡‡fd†}d„}t ˆdˆj j dƒƒ}|j |ƒ|j |ƒ|S(Ns)Can't determine the hash=>id database urlcs;tˆdƒ}|j|ƒ|jƒtjdˆƒdS(Ntws$Downloaded hash=>id database from %s(topentwritetclosetloggingtinfo(tdatat hash_id_db_fd(turlthash_id_db_filename(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytfetch_okzs  cSs$|j}tjdt|ƒƒdS(Ns'Couldn't download hash=>id database: %s(tvalueR6twarningtstr(tfailuret exception((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyt fetch_error€s  tcainfotssl_public_key(R*tostpathtexistst_get_hash_id_db_base_urlR6R>R?tbasenameRt_configtgetR(t addErrback(R;tbase_urlR<RBR+(R(R:R;s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytfetch_itgs        (t_determine_hash_id_db_filenameR((RRNR+((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR!Ys%  cCsc|jjdƒ}|sR|jjdƒs.dStj|jjjdƒdƒ}n|jdƒdS(Ntpackage_hash_id_urlR:t/shash-id-databases(RJRKR*turlparseturljoinR:trstrip(RRM((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRHs cCsØddlm}g}tjj|jƒr>|j|jƒntjj|jƒr”|jgtj |jƒD]}tjj |j|ƒ^qlƒnx=|D]5}t j ƒtjj |ƒ}||j kr›tSq›WtS(s=Return a boolean indicating if the APT sources were modified.iÿÿÿÿ(tPackageMonitor(t landscape.monitor.packagemonitorRURERFRGtsources_list_filenametappendtsources_list_directorytextendtlistdirtjointtimetgetmtimet run_intervalR0R(RRUt filenamestfilenametseconds_since_last_change((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyt_apt_sources_have_changed¡s8 cCsrtjj|jƒr!|j}n+tjj|jjƒrH|jj}ntStj|ƒj}||t j ƒkS(s+Check if the apt-update timeout has passed.( RERFRGtupdate_notifier_stampRJtupdate_stamp_filenameR0tstattst_mtimeR](Rtintervaltstampt last_update((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyt_apt_update_timeout_expired·s cszˆjjs-ˆjƒs-ˆjˆjjƒrXtˆjƒ}‡fd†}|j|ƒStj dˆjƒt dƒSdS(s€Run apt-update and log a warning in case of non-zero exit code. @return: a deferred returning (out, err, code) cs |\‰‰‰d }tˆjjƒtjdˆjˆˆˆfƒˆdkr¡tjdˆjˆˆfƒx^|D]%}|ˆkrud \‰‰‰PququWn.ˆjjƒsÏd‰dˆj ˆj f‰nˆj j d ˆj ˆˆƒ}|j‡‡‡fd †ƒ|S( Ns8Problem renaming the file /var/cache/apt/srcpkgcache.bins5Problem renaming the file /var/cache/apt/pkgcache.bins/'%s' exited with status %d (out='%s', err='%s')is'%s' exited with status %d (%s)tis0There are no APT sources configured in %s or %s.spackage-reporter-resultcs ˆˆˆfS(N((tignore(tcodeterrtout(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRís(s8Problem renaming the file /var/cache/apt/srcpkgcache.bins5Problem renaming the file /var/cache/apt/pkgcache.bin(RlRli(RRJReR6tdebugtapt_update_filenameR>t_facadet get_channelsRWRYR-tcall_if_acceptedt send_resultR((t.0RpRoRntaccepted_apt_errorsR1tdeferred(R(RnRoRps>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR)Îs.       s/'%s' didn't run, update interval has not passedRliN(RlRli( RJtforce_apt_updateRcRktapt_update_intervalRRrR(R6RqR(RR+R)((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR Ãs"   cCs(idd6|d6|d6}|j|ƒS(sP Report the package reporter result to the server in a message. spackage-reporter-resultttypeRnRo(R.(RRnRoR1((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRv÷s  cCsV|j}|ddkr/t|_|j|ƒS|ddkrRt|_|jƒSdS(NR|s package-idst resynchronize(R8R0R't_handle_package_idst_handle_resynchronize(RttaskR1((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyt handle_tasks    csÿg}y|jj|dƒ‰Wntk r:tdƒSXi}xJtˆj|dƒD]2\}}|dkr€|j|ƒqX||| id translations, %d hashes are unknown.cs ˆjƒS(N(tremove(R(trequest(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR&s(t_storetget_hash_id_requestR RR*tzipthashesRXt set_hash_idsR6R7tlent_handle_unknown_packagesR((RR1tunknown_hashesthash_idsthashtidR+((R„s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR~ s$  #   cCsK|jjƒ|jjƒ|jjƒ|jjƒ|jjƒtdƒS(N(R…tclear_availabletclear_available_upgradestclear_installedt clear_lockedtclear_hash_id_requestsRR*(R((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR*s      c Cs<|jjƒt|ƒ}g}g}x¾|jjƒD]­}|jj|ƒ}||kr5|j|ƒ|jj|ƒ}|ji |jd6|jd6|j d6|j d6|j d6|j d6|j d6|jd6|jd 6ƒq5q5W|r,tjd t|ƒƒid d6|d 6}|j||ƒ}n tdƒ}|S( NR|tnametversiontsectiontsummaryt descriptiontsizesinstalled-sizet relationss@Queuing messages with data for %d packages to exchange urgently.s add-packagestpackages(Rstensure_channels_reloadedtsett get_packagestget_package_hashRXtget_package_skeletonR|R•R–R—R˜R™Rštinstalled_sizeR›R6R7RŠt"_send_message_with_hash_id_requestRR*( RRˆt added_hashesRœtpackageRŽtskeletonR1R+((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR‹3s4               cs tjƒ‰ˆt‰‡‡fd†}g}xe|jjƒD]T}|jdkr`|jƒq>|jj|jƒ}|j ||ƒ|j |ƒq>Wt |ƒS(Ncs2|rˆ|_n|jˆkr.|jƒndS(N(t timestampRƒ(t is_pendingR„(tnowttimeout(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytupdate_or_removeZs ( R]tHASH_ID_REQUEST_TIMEOUTR…titer_hash_id_requestst message_idR*RƒR-tis_message_pendingR(RXR(RR«tresultsR„R+((RªR©s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR$Vs   cCs|jjƒtƒ}x]|jjƒD]L}|jj|ƒ}|jj|ƒdkr&|j|jj|ƒƒq&q&Wx*|jj ƒD]}|t|j ƒ8}q†W|s¸t dƒ}nSt |ƒ}|t }tjdt|ƒƒidd6|d6}|j||ƒ}|S(s¸Detect available packages for which we have no hash=>id mappings. This method will verify if there are packages that APT knows about but for which we don't have an id yet (no hash => id translation), and deliver a message (unknown-package-hashes) to request them. Hashes previously requested won't be requested again, unless they have already expired and removed from the database. sBQueuing request for package hash => id translation on %d hash(es).sunknown-package-hashesR|RˆN(RsRRžRŸR R…t get_hash_idR*taddR­RˆRtsortedtMAX_UNKNOWN_HASHES_PER_REQUESTR6R7RŠR£(RRŒR¥RŽR„R+R1((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR%ps&          cs\|jj|ƒ‰ˆj|d<|j|ƒ}‡fd†}‡fd†}|j||ƒS(s<Create a hash_id_request and send message with "request-id".s request-idcs |ˆ_dS(N(R®(R®(R„(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytset_message_idŸscsˆjƒ|S(N(Rƒ(R@(R„(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytsend_message_failed¢s (R…tadd_hash_id_requestRR.t addCallbacks(RR1RŒR+RµR¶((R„s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR£™s  cs(‡fd†}ˆjƒ}|j|ƒS(s×Detect all changes concerning packages. If some changes were detected with respect to our last run, then an event of type 'package-data-changed' will be fired in the broker reactor. cs|rˆjjdƒSdS(Nspackage-data-changed(R-t fire_event(R+(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytchanges_detected°s(tdetect_packages_changesR((RRºRy((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR&¨s cCs-|js|jƒr|jƒStdƒSdS(s„ Check if any information regarding packages have changed, and if so compute the changes and send a signal. N(R't_package_state_has_changedt_compute_packages_changesRR*(R((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR»¸s cCsµ|jj}tjj|ƒs"tStjjdƒ}tjj dƒ}||g}|j t j d|ƒƒtj |ƒj }x0|D](}tj |ƒj }||kr…tSq…WtS(sE Detect changes in the universe of known packages. This uses the state of packages in /var/lib/dpkg/state and other files and simply checks whether they have changed using their "last changed" timestamp on the filesystem. @return True if the status changed, False otherwise. sdir::state::statussdir::state::listss %s/*Packages(RJtdetect_package_changes_stampRERFRGR0tapt_pkgtconfigt find_filetfind_dirRZtglobRfRgR(Rt stamp_filet status_filet lists_dirtfilest last_checkedtft last_changed((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR¼Âs    c s׈jjƒtˆjjƒƒ}tˆjjƒƒ}tˆjjƒƒ}tˆjjƒƒ}tƒ}tƒ}tƒ}tƒ}x½ˆjjƒD]¬} ˆjj | ƒ} ˆjj | ƒ} | d k r•ˆjj | ƒr|j | ƒˆjj| ƒr|j | ƒqn |j | ƒˆjj| ƒrA|j | ƒqAq•q•WxWˆjjƒD]F} ˆjj | ƒ} ˆjj | ƒ} | d k rU|j | ƒqUqUW||‰||‰||‰||‰||‰||‰||‰||‰i} ˆrtttˆƒƒƒ| d/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytupdate_currently_knownHs&  N(RsRRžR…t get_installedt get_availabletget_available_upgradest get_lockedRŸR R±R*tis_package_installedR²tis_package_availabletis_package_upgradetget_locked_packagestlistRR³RRR.R6R7RŠR((Rt old_installedt old_availablet old_upgradest old_lockedtcurrent_installedtcurrent_availabletcurrent_upgradestcurrent_lockedR¥RŽRR1R+RÞ(( RÖR×RØRRÚRÛRÜRÙRÝs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR½Üs|                    ' (RRRR tconfig_factoryt queue_nameRrRWRYRR'R,R.R!RHRcRkR RvRR~RR‹R$R%R£R&R»R¼R½(((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR)s4   7   4 #  )   tFakeGlobalReportercBseZdZeZd„ZRS(sb A standard reporter, which additionally stores messages sent into its package store. cCs&|jj|ƒtt|ƒj|ƒS(N(R…t save_messageRRòR.(RR1((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR.ls(RRRR tpackage_store_classR.(((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRòdst FakeReportercBs,eZdZeZdZd„Zd„ZRS(se A fake reporter which only sends messages previously stored by a L{FakeGlobalReporter}. cshtdƒ}|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|j‡fd†ƒ|S(Ncs ˆjƒS(N(R(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR}scs ˆjƒS(N(R!(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR€scs ˆjjƒS(N(R…t clear_tasks(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR‚scs ˆjƒS(N(tsend_pending_messages(R(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR…s(RR*R((RR+((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR,zs  c s>ˆjdkr"tjdˆ_ntjjˆjƒsAtdƒStˆjj ƒƒ}t ˆjƒ}t|j ƒƒ}||}tdƒ}tƒ}|r:|j |ƒ}g}xs|D]k\} } t j t| ƒƒ} | d|kr¸|j| dƒ|j| ƒ|j| ‡fd†ƒq¸q¸Wˆjj|ƒn|S(sT As the last callback of L{PackageReporter}, sends messages stored. tFAKE_PACKAGE_STORER|cs ˆj|ƒS(N(R.(RR1(R(s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR sN(tglobal_store_filenameR*REtenvironRFRGRRžR…tget_message_idsR tget_messages_by_idsRtloadsR?R²RXR(tsave_message_ids( Rt message_sentt global_storetall_message_idstnot_sentRytgot_typetmessagestsentR®R1((Rs>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyR÷‰s,     N( RRRR RôR*RùR,R÷(((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyRõqs  cCsIdtjkrtt|ƒSdtjkr8tt|ƒStt|ƒSdS(NtFAKE_GLOBAL_PACKAGE_STORERø(RERúR RòRõR(targs((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytmain¥s   cCs8tjjtjjtjdƒƒ}tjj|dƒS(Nislandscape-package-reporter(RERFtdirnametabspathtsystargvR\(R ((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pytfind_reporter_command®s%($RRR6R]R RERÃR¿ttwisted.internet.deferRRtlandscape.lib.sequencerangesRtlandscape.lib.twisted_utilRRtlandscape.lib.fetchRtlandscape.lib.fsRt landscape.libRtlandscape.package.taskhandlerRR R tlandscape.package.storeR R R¬R´R RRòRõRR (((s>/usr/lib/python2.7/dist-packages/landscape/package/reporter.pyts0       ÿÿ= 4