_i^dZddlZddlZddlZddlmZddlmZddlmZm Z  ddl m Z ddl m Z ddlmZdd lmZdd lmZej*ej,d ej.d ej0ej2gej4eZGddZdZedk(reyy#e$r!d dl m Z d dl m Z d dlmZd d lmZd d lmZYwxYw)z? Main orchestration script for Bitbucket SonarQube automation. N)Path)datetime)DictOptional) ConfigManager)BitbucketClient) GitManager)SonarQubeScanner)ReportGeneratorz4%(asctime)s - %(name)s - %(levelname)s - %(message)szsonar_scan.log)levelformathandlersc eZdZdZdefdZdefdZddZde de e fd Z de de fd Zdde d e d e e ddfd ZdefdZddZy)ScanOrchestratorz*Orchestrates the entire scanning workflow.configc*||_|jd}|jd}t|jd|jd|s|nd||_t |jd|_t |jd |jd |jd  |_t|jd |_ t|jd|_ |j|_ y)zt Initialize orchestrator. Args: config: Configuration manager instance bitbucket.api_tokenbitbucket.app_passwordbitbucket.workspacebitbucket.usernameN) workspaceusername app_password api_tokenscanning.repos_dir) repos_dirz sonarqube.urlsonarqube.tokenzscanning.sonar_scanner_path) sonar_url sonar_token scanner_pathscanning.reports_dir) reports_dirzscanning.history_file)rgetr bitbucketr git_managerr sonar_scannerr report_generatorr history_file _load_history scan_history)selfrrrs ,/home/batmin/projects/sonar_scan/src/main.py__init__zScanOrchestrator.__init__*s JJ45 zz":; (jj!67ZZ 45-6D  &jj!56 .jj1 #45$AB  !0 #9:!  !,C!DE ..0returnc|jjr6 t|jd5}tj|cdddSiS#1swYiSxYw#t $r#}t jd|Yd}~iSd}~wwxYw)z!Load scan history from JSON file.rNzFailed to load scan history: )r)existsopenjsonload Exceptionloggerwarningr,fes r-r*zScanOrchestrator._load_historyQs    # # % D$++S1(Q99Q<(( (  D!>qcBCC  Ds4A A A AA A B )BB Nc t|jd5}tj|j|ddddy#1swYyxYw#t $r"}t jd|Yd}~yd}~wwxYw)zSave scan history to JSON file.w)indentNzFailed to save scan history: )r4r)r5dumpr+r7r8errorr:s r- _save_historyzScanOrchestrator._save_history[sj >d''- : $++Qq9 : : : > LL8< = = >s3A#AAA  A A A;A66A; repo_slugc||jvr6|j|jd}|r tj|Sy#t$rYywxYw)z Get last scan date for a repository. Args: repo_slug: Repository slug/name Returns: Last scan datetime or None last_scanN)r+r$r fromisoformatr7)r,rDdate_strs r-_get_last_scan_datez$ScanOrchestrator._get_last_scan_datecsb )) )((377 DH#11(;;!sA AAct|jjd}|jsyt j j d}|d|d}t|j|}t|dkDS)z Check if a report exists for today's date for the repository. Args: repo_slug: Repository slug/name Returns: True if a report exists for today, False otherwise r"Fz%Y%m%d_z_*.pdfr) rrr$r3rnowstrftimelistgloblen)r,rDr# today_strpatternmatching_filess r-_has_report_for_todayz&ScanOrchestrator._has_report_for_todayvs~4;;??+ABC !!#LLN++H5 Kq 62k..w78>"Q&&r/success report_pathctjj||r t|ndd|j|<|j y)z Update scan history for a repository. Args: repo_slug: Repository slug/name success: Whether scan was successful report_path: Path to generated report N)rFrUrV)rrL isoformatstrr+rC)r,rDrUrVs r-_update_scan_historyz%ScanOrchestrator._update_scan_historysE"113/:3{+( )$ r/c |jjdg}|sgSg}|D]N}t|trH|j drf|j ddj dd}|j d}t|dk(sh|\}}|j||d|idd |d gid |j d rk|j d dj dd}|j d}t|dk(s|\}}|j||d|idd |d d |d gid d|vs|j d}t|dk(s/|\}}d|d|d} |j||d|idd | d gid ]t|tso|jdxs|jd}|jd}|jdxs|jd} |s| st|trd|i}nJ|sHd| vrD| j ddj dd}d|vr|j ddnd} | rd| ini}|j|||xsidd | d gid Q|S)z Get additional repositories from configuration. These can be from different workspaces. Returns: List of repository info dictionaries z scanning.additional_repositorieszgit@bitbucket.org:.git/r?slugclonessh)namehref)rbr_rlinkszhttps://github.com/httpsrbrssh_url clone_urlrN) rr$ isinstancerY startswithreplacesplitrPappenddict) r,additional_repos_configrepos repo_configpathpartsr repo_nameorgrfws_names r-_get_additional_repositoriesz-ScanOrchestrator._get_additional_repositoriess#'++//2TVX"Y&I2N K+s+))*>?&../CRHPPQWY[\D JJsOE5zQ/4, 9 $-$-*0))< '-2K$H*"& & !++,AB&../DbIQQRXZ\]D JJsOE5zQ).Y $-$-*0# '-2K$H-4k$J*"& & K''--c2E5zQ/4, 9$6yk9+T"R $-$-*0))< '-2G$D*"& & K.'OOF3N{v7N 'OOK8 %//)4T  8T!)S1%+Y$7 &/7:#*??3G#L#T#TU[]_#`Dr0ddl'}|jQd?|j6jS|} |jTjW||||jd@A}tjdB||j%||jd>d||jd>r|dz }tjdD|n/|dz }tj#dE|d/|jdFCtjdItjdJtjdKt |tjdL|tjdM|tjdN|tjdIy#t:$r+}tj#d.|d/||dz }Yd}~d}~wwxYw#1swYxYw#t:$rYwxYw#t:$r(}tj#dC|d/|d}Yd}~d}~wwxYw#t:$r?}tj#dG|d/|dH|dz }|j%|dYd}~d}~wwxYw)Oz#Run the complete scanning workflow.z"Starting SonarQube scan automationzFound z$ additional repositories from configzHNo repositories found in workspace or additional repositories configuredNz repositories to checkrrbr_z'Skipping repository with no name/slug: zProcessing repository: rr]z*Repository folder is empty or missing for z , cloning...ra)protocolzNo SSH clone URL found for z , skippingr )rDrgzFailed to clone FTzNo report found for today for z , will scanrrzUpdating repository: z Updating additional repository: zNo changes detected for z$, skipping (report exists for today)zChanges detected for z , updating...zFailed to update rK- z [^a-z0-9_] descriptionr\z(Generating sonar-project.properties for z (project key: )) repo_path project_key project_namez0Failed to generate sonar-project.properties for z: z@Could not extract project key from sonar-project.properties for z Project 'z-' does not exist in SonarQube, creating it...r2zutf-8)encodingzsonar.projectName=)r~rr{z(Successfully created SonarQube project: z (z$Failed to create SonarQube project 'z', but continuing with scanz' already exists in SonarQuberUoutput)rs scan_results sonar_results scan_outputzReport generated: zFailed to generate report for zSuccessfully scanned zScan failed for rBzUnexpected error processing exc_infoz<============================================================z Scan Summary:z Total repositories: z Scanned: z Skipped: z Failed: ),r8infor%get_all_repositoriesrvrPextendr9r$rrr3 get_clone_urlr&clone_or_updaterhtuplerBrZrTrIhas_changes_sinceget_sonar_properties_pathlowerrjresubr'generate_sonar_propertiesr7get_project_keycheck_project_existsr4striprirkcreate_projectdebugrun_scantimesleepget_scan_resultsr(generate_report) r,roadditional_repos scanned_count skipped_count failed_count repo_inforDrr}is_empty_or_missingrg clone_resultis_first_clonehas_today_reportworkspace_slugis_additional_repolast_scan_date has_changessonar_props_pathrr~rrr{r<r;linerrrrVs r-rzScanOrchestrator.run_scans3  89335; <<>  KK&%5!6 77[\ ] LL) * NNe f  fSZL(>?@   S K < 1E!FG % 1 *3*:*:*<&<&aYQWEWD_D_Da@a#'KK"LYKWc de!% < $0 )-(,'A'A)'L$+ &DYK{$[\)2{B)G)K)KF)S-;t{{Od?e-e*1"KK*? {(KL"KK*J9+(VW*3{B)G)K)KF)S-;t{{Od?e-e*1-1-E-Ei-PN*...*J*J9Vd*eK#. & .FykQu,v w - 2 ("KK*? {-(XY"KK*J9+(VW!% < $0 ).$(#3#3#M#Mi#X ' $ 0E FI%.Kq "<"B"B"D"L"LSRU"V"^"^_bdg"hK"$&&["IK$-==#8#EIL"+-- r"B"HbKKK"J9+Udepdqqr st !+/+=+=+W+W&/(3)5,X,(#00@@AQR "NN%efoep#qr!Q&M))>>{KKK)K=8e fg$-==#8#EIL"+-- r"B"HbK!"2C'J*a()*#'::<#:#:;N#O37::c13Ea3H3N3N3PL$)**))88$/%1$/9  &N{m[]^j]kkl$mn)Mk]Zu'vwLL9[M9V!WX $11::9FVW #00@@AQR $ <#3#3I#>JJqM$($6$6$G$G $TM '"&"7"7"G"G"+%1&3$0$4$4X$> #H#K KK"4[M BC )) $$Y6  ##I.!Q&MKK"7 { CD A%LLL#3I;bAQAQRYAZ@[!\]]S SXh5G&'s't'0cDxX[}hl ]d]~tr/rc t}|jds*tjdt j dt |jd}t |jd}|jds*tjdt j d|s,|s*tjdt j d|jd s*tjd t j d|jt|}|jy#t$r-tjd t j dYyt$r9}tjd |d t j dYd}~yd}~wwxYw)zMain entry point.rz"Bitbucket workspace not configuredr rrrz'Bitbucket username/email not configuredzOBitbucket authentication not configured (need either API token or app password)rzSonarQube token not configuredzScan interrupted by userz Fatal error: TrN) rr$r8rBsysexitr save_configrrKeyboardInterruptrr7)r has_api_tokenhas_app_password orchestratorr<s r-mainrsB%zz/0 LL= > HHQKVZZ(=>?  +C DEzz./ LLB C HHQK%5 LLj k HHQKzz+, LL9 : HHQK (/   ./   }QC(4 8  sEE 3GG/F<<G__main__)rrr5loggingpathlibrrtypingrrconfig_managerrbitbucket_clientrr&r sonarqube_scannerr r(r ImportError basicConfigINFO FileHandler StreamHandlerstdout getLoggerrr8rrrr/r-rs ! 2,0&20 ,, A,-cjj)   8 $JJZ'T zFY2-1'31 2sB::#C C