_i;dZddlZddlZddlZddlZddlmZddlmZddlm Z m Z  ddl m Z ddl mZddlmZdd lmZej(ej*d ej,ej.g ej0eZd e de e fdZd e de e de e ddfdZddZdedededede e de e de e ddfdZ!edk(r eyy#e$rd dl m Z d dl mZd dlmZd d lmZYwxYw#e"$r&ejGdejHd Yye%$r0Z&ejOde&ejHd YdZ&[&ydZ&[&wwxYw)z Weekly scan: Bitbucket repos with commits in the last N days, pull and run SonarQube scan. Generates a summary report of all pulled and scanned repos (no per-repo PDFs). N)Path)datetime)DictList) ConfigManager)BitbucketClient) GitManager)SonarQubeScanner4%(asctime)s - %(name)s - %(levelname)s - %(message)s)levelformathandlersconfigreturnc |jdg}|sgSg}|D]N}t|trH|jdrf|j ddj dd}|j d}t |dk(sh|\}}|j||d|idd |d gid |jd rk|j d dj dd}|j d}t |dk(s|\}}|j||d|idd |d d |d gid d|vs|j d}t |dk(s/|\}}d|d|d} |j||d|idd | d gid ]t|tso|jdxs|jd}|jd}|jdxs|jd} |s| st|trd|i}nJ|sHd| vrD| j ddj dd}d|vr|j ddnd} | rd| ini}|j|||xsidd | d gid Q|S)z@Get additional repositories from config (same logic as main.py).z scanning.additional_repositorieszgit@bitbucket.org:.git/slugclonessh)namehref)rr workspacelinkszhttps://github.com/httpsrrssh_url clone_urlrN) get isinstancestr startswithreplacesplitlenappenddict) radditional_repos_configrepos repo_configpathpartsr repo_nameorgrws_names 3/home/batmin/projects/sonar_scan/src/weekly_scan.py_get_additional_repositoriesr3!s$jj)KRP " E.8 k3 '%%&:;"**+?DLLVUWX 3u:?+0(IyLL ) )&,i%8")UK,P+Q!R " ''(=>"**+@"EMMfVXY 3u:?%*NCLL ) )&,c]#). D)0+ F&" "  ##))#.u:?+0(Iy 29+QykNGLL ) )&,i%8")UG,L+M!N "  T *#/J;??63JI# 4I!ooi0PKOOK4PGWi-!' 3I"';w'F"??+?DLLVUWXD474Kdjjoa0TG5< 1"I %%!*b%(H'IJ g8r L all_repos to_processc|jdxsg}t|tr=|jdDcgc]#}|j s|j %}}|sy|Dchc]H}|jds|jdr$|jdxs|jdJ}}|D]}|r||vr |D]g}|jdxs|jd} | |k(s-|j ||j |tjd|utjd|ycc}wcc}w)Nz#scanning.scan_recent_always_include,rrz-Including repo (scan_recent_always_include): ucscan_recent_always_include: slug %r not in repo list — add scanning.additional_repositories entry) r!r"r#r&stripr(addloggerinfowarning) rr5r6rawsrseenr repo_inforslugs r2!_merge_scan_recent_always_includerDds8 **: ; ArC#s"%))C.>QAGGIqwwy>>   55=AEE&M v '!%%- ' D  tt| " IMM&)BY]]6-BE}!!), KD6RS  NNu ? sE E *A Ec ttjjj}t |t j vr)t j jdt |tj||dz }|jddtjjd}|d|dz }tj|d }|j!tj"d tj$j'|t(j+d  t-}j5ds*t(j1dt j2d t7|j5d}t7|j5d}|j5ds*t(j1dt j2d |s,|s*t(j1dt j2d |j5ds*t(j1dt j2d |j5dxsd} |j5d} t|j5d} t|j5d} | jddt9| |j5d|s|j5dnd|j5d} t;|j5d}t=|j5d|j5d|j5d }| j?xsg}tA|}|r3t(j+d!tC|d"|jE||s't(jGd#tI|| || gggy|j5d$xsg}tK|tr=|jMd%Dcgc]#}|jOs|jO%}}|r|Dchc]}|jQ}}|Dcgc]=}|j5d&xs|j5d'xsd(jQ|vs<|?}}t(j+d)tC|d*d+jS|d,|s't(jGd-tI|| || gggyg}|D]}|j5d&xs|j5d'}|s*|j5d.xsij5d'}|r/|| k7r*|jU|t(j+d/|| jW|| 0s|jU|t(j+d1| d2|tY|||t(j+d3tC|g}|D]}|j5d&xs|j5d'}| j[|d45}|s.t(jGd6|d7|jU|d8d9fj |j]||:}|xrtK|t^xr|d}|jU|||rd;nd|g}|D]}|j5d&xs|j5d'}| |z }|jcr|dAz jcs|jU|d8dBfe|je|} | s|j5d.ij5d'xs| }!|!dC|jQjgdDdCjgdEdC}"tijjdFdC|"}"|j5d&xs|}# |jm||"|#G} |jo| }"|"s|jU|d8dIf<|jq|"s?|j5d&xs|}#|j5dJd(xsd(}$|js|"|#|$K |ju|| }|j5dLd8}%|%s|j5dMd(nd(}&|jU||%|&xs|%rd;ndNf|%rt(j+dO|nt(jGdP|dQ|&tI|| || |||t(j+dSy#t.$r8}t(j1d |t j2d Yd}~d}~wwxYwcc}wcc}wcc}w#t.$rC}t(jad?|d@||jU|d8t |fYd}~d}~wwxYw#t.$rC}t(j1dH|d@||jU|d8t |fYd}~,d}~wwxYw#t.$rC}t(jadR|d@||jU|d8t |fYd}~{d}~wwxYw)Tz"Run weekly pull and scan workflow.rlogsT)parentsexist_okz %Y%m%d_%H%M%S weekly_scan_z.logutf-8encodingr zAStarting weekly pull and scan (repos with commits in last N days)zFailed to load config: r Nzbitbucket.workspacez"Bitbucket workspace not configuredzbitbucket.api_tokenzbitbucket.app_passwordzbitbucket.usernamez'Bitbucket username/email not configuredzHBitbucket authentication not configured (need API token or app password)zsonarqube.tokenzSonarQube token not configuredzscanning.weekly_scan_dayszscanning.repos_dirzscanning.reports_dir)rusername app_password api_token) repos_dirz sonarqube.urlzscanning.sonar_scanner_path) sonar_url sonar_token scanner_pathzFound z$ additional repositories from configzNo repositories foundzscanning.weekly_scan_reposr8rrrzLimited to whitelist: z repos (z, )z0No repos matched the weekly_scan_repos whitelistrz-Including additional repo (no commit check): )daysz Including repo (commits in last z days): zRepos to pull and scan: r)protocolzNo SSH clone URL for z , skippingFz No clone URL) repo_slugr OKz Pull failedz Pull OK: z Pull failed: zPull error for : rzRepo missing or not a git repo_- z [^a-z0-9_]) repo_path project_key project_namez0Failed to generate sonar-project.properties for z*No project key in sonar-project.properties description)r_r`rasuccesserrorz Scan failedz Scan OK: z Scan failed:  - zScan error for z)Weekly scan finished. See summary report.);r__file__resolveparentr#sysr-insertoschdirmkdirrnowstrftimelogging FileHandler setFormatter Formatter getLogger addHandlerr;r<r Exceptionrcexitr!boolrr r get_all_repositoriesr3r'extendr=_write_summaryr"r&r9lowerjoinr(has_commits_in_last_n_daysrD get_clone_urlclone_or_updatetuple exceptionexistsget_sonar_properties_pathr%resubgenerate_sonar_propertiesget_project_keycheck_project_existscreate_projectrun_scan)' project_rootlog_dirlog_tslog_file file_handlerre has_api_tokenhas_app_passwordrVmain_workspacerQ reports_dir bitbucket git_manager sonar_scannerr+ additional whitelistr? whitelist_setr@r6rBrXws_slug pull_resultsr resultok scan_resultsr^sonar_props_pathrr_r`rarberrs' r2run_weekly_scanrs? >))+2299L <( 3|,-HH\V#G MM$M. \\^ $ $_ 5F<xt44H&&x'BLg//0fgh ""<0 KKST ::+ , 9:  $9:;MFJJ'?@A ::* + >?  !1 _`  ::' ( 56  ::1 2 7aDZZ 56NVZZ 456Ivzz"89:KdT2  01ANVZZ 89TX**23 I vzz2F'GHK$**_-JJ01ZZ =>M  * * , 2E-f5J fS_--QRS Z ./xfdBBG 78>BI)S!(1(<J1 QWWYJ J,56q6 6!eqaeeFm&JquuV}&J%Q%Q%SWd%dee ,SZL9AU@VVWXY NNM N 8[&$B K J V MM&)BY]]6-B  ==-388@ w.0   i ( KKG {S T  1 1)$ 1 G   i ( KK:4& T U V&feZ@ KK*3z?*;<=L< MM&)BY]]6-B ++I+F  NN29+ZH I   E> B C  < 009PY0ZFEZ6D6!9B   B N O i {34yk:;<(L1< MM&)BY]]6-B  ) !)f*<)D)D)F   E3S T U &@@K! k26::6BTnI&Kq 4::<DDS#NVVWZ\_`K&&[AK$==0=IL #0#J#J' +!-$K$ $334DE    E3_ ` a 11+>$==0=IL#-- r:@bK  ( (')' )  <"++I7GHFjjE2G18&**Wb)bC   GS5`WTR_ a b i {34ykSEBC]1? 8 <   ykA3? @   E3q6 : ; ; z!_write_summary..K72qB!7 z rYFAILrZrdzPulled: rz Scan results:c3.K|] \}}}|s dywrrrs r2rz!_write_summary..Orrz Scanned:  rJrKzSummary written to z Log file: ) rrm isoformatr'sumr(ryr| write_textr;r<)rrrrVr6rr summary_pathlinespull_okrXrmsgscan_ok summary_texts r2rzrz7s#7r |4!HHL !9!9!; <= $u% (Z(9:  E7L77G*I 2s r"$&1I;c#GHI LL" 3|+<*=>OTU7L77G*I 2s r"$&1I;c#GHI LL" '!C ,=+>?JK99U#LL7; KK%l^45 KK*XJ'(r4__main__zInterrupted by userz Fatal error: )rN)(__doc__rjrrhropathlibrrtypingrrconfig_managerrbitbucket_clientrrr sonarqube_scannerr ImportError basicConfigINFO StreamHandlerstdoutrs__name__r;r3rDrr#intrrzKeyboardInterruptr<rvrurrrr4r2rs  4,0&2 ,, A#g##CJJ/ 0   8 $@@4:@FmT `dei`jos8t=n))) )  ) T ) u+ )u+) )D zO 4-1'3 4T  )* =,- s/CC<C98C9<(E&E*&EE